Dimensions of Impact in the Social Sciences: The Case of Social Policy, Sociology and Political Science Research 9781447327943

Citation preview

TINA HAUX

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES The Case of Social Policy, Sociology and Political Science Research

POLICY PRESS

RESEARCH

First published in Great Britain in 2019 by Policy Press North America office: University of Bristol Policy Press 1-9 Old Park Hill c/o The University of Chicago Press Bristol 1427 East 60th Street Chicago, IL 60637, USA BS2 8BB UK t: +1 773 702 7700 t: +44 (0)117 954 5940 f: +1 773 702 9756 [email protected] [email protected] www.policypress.co.uk www.press.uchicago.edu © Policy Press 2019 British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library. Library of Congress Cataloging-in-Publication Data A catalog record for this book has been requested. ISBN 978-1-4473-2408-9 ISBN 978-1-4473-2795-0 ISBN 978-1-4473-2796-7 ISBN 978-1-4473-2794-3

(hardback) (ePub) (Mobi) (PDF)

The right of Tina Haux to be identified as author of this work has been asserted by her in accordance with the Copyright, Designs and Patents Act 1988. All rights reserved: no part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise without the prior permission of Policy Press. The statements and opinions contained within this publication are solely those of the author and not of the University of Bristol or Policy Press. The University of Bristol and Policy Press disclaim responsibility for any injury to persons or property resulting from any material published in this publication. Policy Press works to counter discrimination on grounds of gender, race, disability, age and sexuality. Cover design by Policy Press Front cover image: Pchyburrs Printed and bound in Great Britain by CPI Group (UK) Ltd, Croydon, CR0 4YY Policy Press uses environmentally responsible print partners

Contents

Lists of tables and boxes

iv

Acknowledgements vi one Introduction

1

two

Dimension 1: The role of academic research in policymaking

7

three

Researching impact

21

four

Dimension 2: The nature of the underlying research and characteristics of the researchers

35

five

Dimension 3: The nature and reach of impact

61

six

Dimension 4: Taking the long view: looking back over 40 years of Social Policy

83

seven

Summary and conclusions

101

Appendix

109

References

121

Index

135

List of tables and boxes

Tables 3.1 4.1 4.2 4.3 4.4 4.5 5.1 5.2 5.3 A1 A2

Number and ranking of impact case studies included in the sample by sub-panel Nature of the research methods Number and range of publications based on references to research Research funding and set-up Main characteristics of the applicant/s Impact case study submission by type of university Type of impact Linking nature of research and impact Impact agents and pathways Ranking for Impact Case Studies relative to overall ranking Panel membership

30 36 43 44 47 49 72 73 76 113 114

ICS based on quantitative research ICS based on qualitative research ICS based on conceptual research Accompanying technical change Contentious and/or interdisciplinary topics The introduction of policies rather than their effects/effectiveness? The issue of collaboration and attribution

38 40 41 51 53 54 56

Boxes 4.1 4.2 4.3 4.4 4.5 4.6 4.7

iv

List of tables and boxes

4.8 4.9 5.1 5.2 5.3 5.4 5.5 5.6 5.7

Infrastructure Public engagement Examples of policy innovation Examples of new policy directions Examples of new policy discourse Examples of changes in practice Examples of new data/knowledge capture Sustained and multi-faceted engagement as a pathway to impact A direct impact on people’s lives

v

58 58 64 66 67 68 70 77 79

Acknowledgements

This book has been long in the making and, therefore, there are many people to whom I would like express gratitude for making it happen. A big thank you, then, to Lucinda Platt, who encouraged me to write a book; to Laura Vickers and Catherine Gray for being such supportive editors at Policy Press; to the reviewers and professional staff at Policy Press; to Hugh Bochel and my lovely former colleagues at the University of Lincoln, who provided me with encouragement and internal funding to get started; to my accountability group, Beth Breeze and Ben Baumberg Geiger, for your advice on how to say no; to my writing buddies, Renee Luthra and Mike Brewer, for sharing the highs and lows of the writing process; and to my family, Ben, Karl and Lukas, for giving me precious time to write this book. Finally, the biggest thanks go to the respondents, who shared their time and thoughts so generously with me. The memories of the conversations, the hospitality and the commitment to improving people’s lives have been the best part of the project.

vi

ONE Introduction

Universities are ivory towers no longer (if ever they were) – academics are now increasingly required to demonstrate their contributions to the wider world. Often referred to as ‘impact’, this term is now understood to mean any instance where academic research produces something other than pure knowledge, and so has some influence on the economy or society, or political and cultural life. Impact has become a popular additional measurement of the ‘usefulness’ of research for research funders internationally and, since 2014, the governments in the UK. ‘Usefulness’ of research in this context is based on the increasing demand for detailed accounting and justification of public spending but also international competitiveness based on collaborations between universities and industry (Hicks, 2012). The inclusion of impact as an assessment category in the 2014 Research Excellence Framework (REF2014) in the UK marked the first time that the work of academics was going to be assessed on criteria beyond the research environment and publications as measured by peer review. The UK led the way with the introduction of the Research Selectivity Exercise in 1986. Similar performance-based research funding systems have since been introduced in ten countries (Hicks, 2012; see also Jonkers

1

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

and Zacharewicz, 2016, and Zacharewicz et al, 2018). While the first Research Selectivity Exercise in 1986 was regarded as relatively ‘light touch’ with only five outputs required per department (Gilroy and McNamara, 2009). Subsequent research assessment exercises in the UK, carried out every three to four years, became increasingly transparent and time-consuming (HEFCE, 2008). The inclusion of impact in the REF2014 was greeted with scepticism by many academics, who highlighted the difficulties in defining, measuring and attributing impact (see Martin, 2011). Many of these criticisms reflect the difficulties faced by academics trying to influence policy-making more generally and thus can be found in the literature on evidence-based policymaking (EBPM) since the 1960s (for the most recent iterations see Head, 2016; Cairney, 2016; Parkhurst, 2017; French, 2019). Key barriers are timeliness, fit, trust/support and communication. More appropriate communication, the use of knowledge brokers as well as ongoing engagement with policymakers are the most frequently cited solutions to the problem (see Cairney, 2016; Reed, 2016; French, 2019). However, policymaking relies on bringing different coalitions of knowledge, experience and values together and academic research only forms one part of that. Despite the advance in scholarship on the link between research and policymaking, the quote by Lindblom and Cohen (1979: 1) that, ‘in public policy making, many suppliers and users of social science research are dissatisfied, the former because they are not listened to, the latter because they do not hear much they want to listen to’ captures the sentiments of many on both sides. This may not be the time, however, to doubt the ability of academics to influence policymaking, because being able to produce successful impact narratives has become a currency for universities and individuals as part of that more balanced scorecard. It is therefore perhaps not surprising that the immediate question I get when mentioning this book to my

2

Introduction

academic colleagues, is, ‘So, tell me, what is impact?’ Many of my colleagues are in the middle of putting their own impact case studies (ICS) together and are therefore curious about whether their case would count as impact and, if so, how to present it. It is telling that five years after the REF2014, there is still confusion over what constitutes impact, particularly in the social sciences. Hopefully, this book will be able to provide a partial answer to this on the basis of analysing ICS from the 2014 REF as well as interviews with academics looking back over their career and evaluating the impact they have (or have not) had. At this stage, much has been written about impact. The book aims to add to this literature by investigating four dimensions of impact in the social sciences. These four dimensions are the policy context, the nature of the underlying research and characteristics of the researchers, the reach and significance of the impact and, finally, its significance over time. The first dimension places impact in the large, and largely critical, literature of the role of academic research and researchers in the policy-making process more generally (see Chapter two). Chapter three focuses on how impact as part of the REF2014 has been evaluated and researched before setting out the research design of this study. Building on the work by Bastow et  al 2014, Smith and Stewart 2016 and Dunlop 2018 in particular, the second dimension captures the nature of the underlying research across the different social science disciplines included in this book (Chapter four). The analysis compares and contrasts the methodology, publication outlets, author characteristics and funding of the ICS as well as coming back to the issues around the ability to have and to measure impact raised in Chapter two such as timeliness, attribution and the nature of the policy problem. The third dimension tries to capture the nature and reach of impact by developing a typology of impact (see Haux 2018 and Chapter five). The typology consists of five groups: new policies, new policy directions, new policy discourse, changing practice and new knowledge/data capture. For the first group, new policies,

3

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

the research has directly led to a new policy being introduced. In the second group, policy direction, the claim is that the work of the academic/s has shaped part of a policy rather than instigated a new policy. The third, policy discourse, research has affected the framing of a group or an issue as well as having contributed different policy options to existing discourse. The fourth set of case studies is based on change in policy practice of professionals, mostly social workers and the police. The final set of ICS are catagorised under new knowledge and data capture. This refers to contributions such as new indicators or other ways of capturing data. The fourth, and final, dimension of impact covered in this book is its contextualization over time and, therefore, a more critical perspective on the ability of academics to have impact based on interviews with 16 Social Policy professors (Chapter  six). These interviews provide a critical framing of impact by drawing attention to the time lags that go well beyond the impact timetable given in the REF, the often partial adoption of research, the use of knowledge brokers and, perhaps what is more important, a clear sense of being part of an epistemic community and a clear sense of ambition for the scale of impact and the nature of the relationship with policymakers. Chapter seven brings together the key findings from the different chapters and looks ahead to the REF2021. The contribution of this book is threefold: unlike most other studies of the REF2014 ICS, this book includes case studies from three different sub-panels, each of which cover several academic disciplines. Therefore, it allows for an in-depth comparison across three key social science disciplines: Social Policy and Social Work, Sociology and Politics and International Relations. Second, the ICS are placed in an analytical framework that identifies different types of impact and impact pathways and places them in the context of academic identities and different policy models. Finally, the current form of capturing impact in the REF2014 is contextualised by providing a comparison over time based on interviews with Social Policy professors who are

4

Introduction

looking back over 40 years at their careers, as well as analysing the relationship between research and policymaking. This long view allows the achievements as well as the serendipitous and superficial nature of impact to be put into the context of different governments and stages of welfare state development. Future research assessment exercises will show the effects and effectiveness of impact more clearly given the collective learning that has taken place since the 2014 REF (Martin, 2011; Derrick, 2018). Hopefully, the analysis of the impact case studies of the REF2014 in this book will contribute to that learning.

5

TWO Dimension 1: The role of academic research in policymaking

Including impact as part of the assessment of academic excellence in the UK has forcefully drawn the attention of universities to the relationship between research and policymaking. ICS will also form part of the forthcoming 2021REF and ‘doing impact’ has now become part of the workload and performance goals of many academics. While the central tenet that academics do have influence on the world around them is not disputed, the necessity and ability of capturing said effects has been questioned and the resulting criticisms will be discussed later in this chapter. Yet, the relationship between research and policymaking has been the subject of much debate long before the REF2014 (among others Blume, 1977; Weiss, 1979; Nutley et al, 2007; Head, 2008; Cairney, 2016). Much of it has been located in the field of policy analysis and the focus has been on the extent to which policymaking has been based on and informed by research. Thus, in contrast to the REF, EBPM places policies and policymaking at the centre of the analysis rather than the work by individual/groups of researchers. The aim of this chapter is to bring the two discussions together, i.e. to discuss

7

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

the relationship between academic research and policymaking by placing the expectations and criticism of impact as part of the REF2014 in the broader literature of EBPM. REF2014: measuring the impact of academics The inclusion of impact as an assessment category in the REF2014 in the UK marked the first time that the work of academics was going to be assessed against criteria beyond the research environment and publications (HEFCE, 2012). Adding impact as an additional measure of academic excellence was mentioned in the review of the research assessment exercise in 2006 (HEFCE, 2012, see also Williams and Grant, 2018). The stated rationale for adding impact to the REF2014 was to ‘develop and sustain a dynamic and internationally competitive research sector…that makes a contribution to economic prosperity, national wellbeing and the expansion and dissemination of knowledge’ (in HEFCE, 2009: 38, cited in Williams and Grant, 2018: 97) and to judge universities using a ‘balanced scorecard’ (Smith et al, 2011: 151– 152).For the purposes of the REF, impact has been defined as the ‘reach and significance’ of impacts on the economy, society and/or culture that were underpinned by excellent research conducted in the submitted unit, as well as the submitted unit’s approach to enabling impact from its research. (HEFCE, 2014a: 6) HEFCE gave further guidance on what counted as impact, namely, Impact includes, but is not limited to, an effect on, change or benefit to: • the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding

8

Dimension 1: The role of academic research in policymaking

• of an audience, beneficiary, community, constituency, organisation or individuals • in any geographic location whether locally, regionally, nationally or internationally. Impact includes the reduction or prevention of harm, risk, cost or other negative effects. (HEFCE, 2012: 26) Following a pilot (REF 2014) exercise, it was decided that impact was to be assessed on the basis of qualitative case studies submitted on an impact template (King’s College London and Digital Science 2015), which included the following subheadings: summary of impact, underpinning research, references to the research, details of the impact, and sources to corroborate the impact (HEFCE, 2014a). An ICS could be no longer than four pages; one ICS was required per ten FTE submitted staff; and the underlying research needed to be of two-star quality or above (HEFCE, 2014a). The timing of the impact was to be the same as the assessment period for the publications and the environment (2008–2013). However, the research forming the basis of the impact could be from further back (it could date as far back as 1993) in order to allow for a time lag between research and impact (see Derrick, 2018 for a discussion of the ‘evaluation mechanics’ of impact). Criticism of how impact is defined and measured in the REF The initial announcement of the addition of impact to the measurement of academic work was greeted with scepticism, at best, by the academic community. Since then, the notion of measuring external influence has become more accepted not just as a necessary evil, but as an exercise that allows academics to showcase their wider contribution. Nevertheless, the debate over the definition, attribution and measurement of impact (among others, Holmwood, 2011; Brown, 2012; Bornmann,

9

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

2013; Penfield et al, 2014), the escalating cost of preparing a REF submission for departments (Sayer, 2015), the impact on individual careers when used as managerialist tool (Docherty in Morrish, 2014), as well as an incentive to focus on current, safe, policy-relevant issues at the expense of more radical and potentially uncertain, blue sky thinking (Smith, 2010, and Smith and Stewart, 2016) has not gone away. Moreover, the emphasis on academics having to account for their research focus fundamentally undermines academic freedom (Smith et al, 2011). The current conceptualisation and suggested measurement of impact as part of the REF suggests ‘a linear model of how knowledge from an individual piece of research is subsequently taken up and used’ (Martin, 2011: 250). Some of the other criticisms are perhaps obvious but no less valid. These are that impact can be ‘coincidental, even serendipitous’ (Brewer, 2011: 256), that it can be negative (the study on the supposed link between autism and the MMR vaccine being perhaps the most famous recent example), that it can depend on the political leaning and agenda of respective governments (Brewer, 2011), that it does not sufficiently account for the time lag and cumulative nature of research and, thus, creates problems of attribution (Penfield et al, 2014) and that it does not capture excellence in teaching and curriculum development (Wade, 2013). Furthermore, a similar pattern is likely to emerge with regard to the impact as with other elements of the research assessment exercises. Namely, that the next attempt will become more transparent but also more cumbersome in response to criticisms, and that soon the societal benefits of any ‘impact’ will be outweighed by the cost of the exercise (see Martin, 2011) The costs of the REF2014 has been estimated at £7,500 per case study and £55 million for universities (Morgan Jones et al, 2013). Already, many universities have employed impact case officers, and have made small grants available for impact activities. Also, while the introduction of the impact assessment

10

Dimension 1: The role of academic research in policymaking

is likely to lead to a change in behaviour within academia, for example, generally more engagement with potential users, the additional benefit of repeat exercises is likely to diminish (Morgan Jones et al, 2013). In essence then, the definition of impact in the REF2014 contained the following elements: • impact is self-defined both in terms of the contribution to a particular change as well as the change itself; • the impact narrative is forward not backward looking, that is, it takes the academic work as the starting point and links it to a policy outcome and not the other way round; • impact is linked to individuals and institutions rather than research collaborations that may span across universities, as in the Australian system (King’s College London and Digital Science, 2015); • impact is presented in the form of individual case studies based on a relatively short template; • the case studies are submitted together with ‘evidence’ from end users serving as support and, to an extent, verification. Examine impact through the lens of evidence-based policymaking ‘The prospect of mutual benefits for managers, researchers and citizens is alluring’ (Head, 2008: 1). The original interest in the 1960s and 1970s in the contribution of the social sciences to policymaking was driven by large investments into social programmes, and the associated rise of New Public Management with its focus on performance management based on indicators from the 1960s onwards. Much of the literature from that time is still relevant to the debates today (for example, Blume, 1977; Pahl, 1977; Caplan, 1979; Weiss,1979; 1982; Bulmer, 2015). The movement, which has since become known as evidencebased policymaking (EBPM), then saw a renaissance in the late 1990s. The aim was to move beyond ideology and identify

11

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

‘what works’ (see Nutley et  al, 2007; Bochel and Duncan, 2007). Despite huge investment in research capacity and programme evaluations, especially in the UK, the assessment of the effectiveness of the research commissioned at the time, including large scale evaluations, in terms of influencing policymaking is sobering in terms of the actual contributions made to policymaking (Bochel and Duncan, 2007). However, French (2019) and Cairney (2016) suggest that there is yet another wave of EBPM enthusiasts arguing that research should inform, or even, drive policymaking more than it does at present. At the core of the literature and practice of EBPM is the relationship between policy and evidence in both its ideal and actual forms. This will be discussed with reference to the underlying policy models as well as barriers to the take-up of research. Starting with the role of research in policymaking, that is, the understanding of how policymaking happens and what the main drivers are, I would like to highlight three different models. The first two are taken from the, now classic, list of seven policy models developed by Weiss (1979; see also Nutley et al, 2007 and Smith, 2013). The first model, and arguably the module underlying a simplified and simplistic interpretation of impact, is the ‘knowledge’ model. As the name suggests, the starting point and main driving force for shaping a policy is research. In many ways the most straightforward model, the argument here is that research can and should guide policymaking and in its purest or perhaps slightly caricatured form this is regarded as a linear relationship whereby a new piece of research gets translated into policy in clear steps (see Cairney, 2016 for a discussion of complete rationality). One of the main distinguishing features of this model is the ability and primacy of research to provide solutions to policy problems. In other words, other evidence such as programme implementation experience, user views or political viability are regarded as secondary. At the other end of the spectrum is the ‘interactive’ model, similar to the network model of policymaking, which emphasises

12

Dimension 1: The role of academic research in policymaking

the involvement and interplay of different actors (see Weiss, 1979; Head, 2008). Those engaged in developing policy seek information not only from social scientists but from a variety of sources – administrators, practitioners, politicians, planners, journalists, clients, interest groups, aides, friends and social scientists, too. The process is not one of linear order from research to decision but a disorderly set of interconnections and back-and-forthness that defies neat diagrams. (Weiss, 1979: 428) One of the reasons for the broad approach to knowledge gathering are the different lenses required for policymaking, namely those of the political, the academic and the practitioner. Each of them has different, complementary knowledge (Head, 2008) such as the awareness of partisanship and bias, the tactical knowledge and the ability to dissect and influence political priorities, ‘systematic approaches to gathering and analysing information’ as well as knowledge of policy implementation (Weiss, 1979: 6). Not included in Weiss’ original list of policy models, the two-communities model suggested by Caplan (1979) is frequently invoked when describing the relationship between policymaking and research. This model highlights the cultural and institutional gaps between policy-making and research (most recently, Cairney, 2016; Stoker and Evans, 2016; Parkhurst, 2017). According to Mead, most academic research on public policy achieves little influence in government. The disconnect reflects the different ways researchers and government learn about policy. In social policy, typically, scholars make rigorous but narrow arguments about how to improve social conditions while saying little about politics or government.

13

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Policymakers, however, reason in broader, integrative ways and pay more attention to program experience and institutions. (2015: 257) Newman and Head (2015) argue for a more nuanced assessment that takes into account factors such as institutional support and the educational background of policymakers and distinguishes between the administrative and the political level of government, that is, the civil servants versus the special advisors in the UK context. Yet, if policymaking is indeed ‘political, not systemic’ and if ‘decisions made in the public sector are the result of efforts to balance competing societal interests, resolve power conflicts, and appease groups with widely divergent values’ (Newman, 2017: 1107), then a linear relationship between research and policy is impossible and, what is more important, the role of the academic can then be at best advisory or perhaps needs to be political in order to gain leverage (Cairney, 2016). Barriers and facilitators of using research Moving on from the underlying conceptualisation of the relationship between policy and research to the barriers to and facilitators of the take-up of research by policymakers:, the key barriers to the use of research by policymakers can be summarised as timeliness, fit, trust/support and communication (see Nutley et  al, 2007; Bochel and Duncan, 2007; British Academy, 2008, and Newman et al, 2016). Timeliness in this case refers to the different timescales of policymaking and research. Policymakers rarely have time to investigate all the possible options (Bochel and Duncan, 2007; Cairney, 2016) and even less so to commission specific research. Academic research is often conducted over substantial periods of time and aims to produce a comprehensive answer to a specific question and thereby adding to our overall knowledge of the subject. Therefore, the answers sought by policymakers at any

14

Dimension 1: The role of academic research in policymaking

point in time regarding a particular policy problem may not be available because the research is either not complete or not conclusive or sufficiently specific to provide the answer sought by policymakers. A number of studies have explored further the dimension of timing of impact within the academic career. Bastow et al (2014) argue that there tends to be a window of opportunity for mid-career researchers as they have (slightly) more time and security and also academic kudos compared to junior researchers, before being drawn back into ‘core’ academic responsibilities such as becoming Head of Department. By the same token, the authors argue that academics have a lack of realism about the possibility to exert influence, especially on policymakers, but also a lack of awareness of the level of time commitment required to do so, which then frequently leads to frustrations and withdrawal from the process (Bastow et al, 2014). Linked to the timeliness is the fit and relevance of research. While academics tend to aim for conclusive answers to specific questions, policymakers often want practical answers to broader policy problems (Bochel and Duncan, 2007; Mead, 2015; Cairney, 2016). Smith (2010) suggests furthermore that academics frequently self-censor their contributions when in the public sphere in order to be seen as trustworthy and constructive while being more outspoken within academic environments. Finally, academics were found to be often modest, ‘diffident or alternatively punctilious’ about the impact of their research, referring to its arbitrary nature and it usually being based on a larger body of work by numerous scholars (Bastow et al, 2014). This can partly be addressed through better communication and translation of research findings into policy proposals as there is often regarded to be a poor ‘fit’ between the academic research and the practical needs of policymakers (Bochel and Duncan, 2007; British Academy, 2008; Head, 2016). However, a gap remains between what is regarded as high quality research in academia and as useful and useable policy insights by policymakers.

15

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Fit does not only capture whether research is sufficient, practical and political (Head, 2016) to be useful for policymakers but also the nature of the research contribution and the nature of the problem. Research is less likely to be taken up if it is critical of the government policies of the day. (Head, 2016). However, more subtly, it is suggested that research is more likely to be taken up in fields where there is less ideological disagreement (Head, 2016) or where the policy field itself is seen as more amenable to objective analysis, such as public health programmes (Head, 2016). Moreover, in some fields, the use of EBPM is far more established than in others, such as healthcare services, children and youth development, family services and education (Head, 2016). This links to research being more likely to influence policy decisions if the contribution is technical. When considering the nature of the policy problem Head talks about a continuum with ‘discrete, bounded’ problems that are ‘linked to a particular set of information and actors’ (Head, 2008: 4). These problems are likely to require a technical solution from experts in this field. However, at the other end of the continuum are ‘complex, inter-linked and cross-cutting’ problems, which therefore call for a diversity of networks and stakeholders to obtain the relevant expertise and buy-in and thus move from the technical to the relational approach. The latter is characterised by multiple evidence bases which ‘inform and influence rather than determine’ policy options (Head, 2008: 4), that is, based on an interactive or network model of policymaking. Regarding trust and support for research, relevant research is often regarded as a low priority by individuals and organisations involved in policy-making such as civil servants (British Academy, 2008; Newman et  al, 2016). The reasons go beyond a lack of time and include institutional support for research, policymakers’ educational background and familiarity with research and, most importantly, attitudes towards research (Newman et al, 2016). Attitudes towards research are driven by

16

Dimension 1: The role of academic research in policymaking

its usefulness, as discussed earlier, but also the trustworthiness of research. ‘Trust’ here refers not so much to quality but to ideological alignment and the perception of an understanding of the policy and political context (British Academy, 2008). Often the most trusted pieces of research are those produced in-house or those specifically commissioned. Yet the ability to dissimilate knowledge written by academics tends to correlate with additional (research-related) education and therefore varies considerably among policymakers and practitioners (see British Academy, 2008; Head, 2016). This applies particularly to quantitative research, given the lack of familiarity with these methods among UK social scientists (British Academy, 2008). The importance of communication is highlighted in the literature on EBPM and it contains a number of suggestions on how to overcome barriers to research use, in particular using knowledge brokers, co-production with policy-makers, and regular engagement between academics and policy-makers (Nutley et al, 2007). The role of the knowledge broker is to act as a conduit who is able to communicate with both academics and policymakers, and is often highlighted as necessary (by, among others, Head, 2016). The concept of a knowledge broker is clearly based on the two-communities model by Caplan outlined earlier, that is, two separate groups of professionals who are not able to communicate with each other effectively without such a conduit. Other recommendations for overcoming the twocommunities problem are for academics to get political (Cairney, 2016), to make better use of the range of existing research methods (Stoker and Evans, 2016), as well as to improve the governance of evidence such as its robustness (Parkhurst, 2017). Another way to address the distance between the two communities is the growing amount of ‘co-production’ research, in which decision-makers are involved from an early stage (see Wehrens, Bekker and Bal, 2011; Smith and Joyce, 2012). However, this again requires longer time-scales, which are not always available when it comes to designing policy

17

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

(or research). Perhaps the main change is that, increasingly, evaluations are commissioned alongside big policy projects such as Sure Start and the New Deal in the UK. These collaborations have not always been happy (see Smith and Haux, 2017), nor is it necessarily the case that even the in-house research is integrated into future research design to avoid asking the same questions time and again (Haux, 2013). Finally, the importance of regular engagement and exchange between policymakers and academics is mentioned by all EBPM scholars (see Head, 2008; 2016). Mead (2015) supports regular engagement but goes further in arguing for professional policymaking experience being valued more highly in academic appointments and, generally, the benefit to both sides from secondments at different career stages. Conclusion The motivation for, ability to, support of and extent to which research is used by policymakers varies considerably. Issues of timing, trust and usefulness in terms of specificity, forward thinking and persuasive power/intentions have hindered the uptake of research. Head summarises much of the research by stating that ‘the early hopes of the evidence-based policy movement for large and rapid improvement of policies and programmes through better use of rigorous research were not rapidly fulfilled’ (2016: 474) and Heinrich goes further by saying that despite advances in our analytical tools and capacity for assembling performance information and scientific evidence, it has become increasingly clear that we are still far from a consensus – intellectually or politically – regarding what should count as evidence, how it should be produced and validated and how it should be used to influence policy. (2007: 259; see also French, 2019)

18

Dimension 1: The role of academic research in policymaking

This has led some to move away from EBPM to evidenceinformed policymaking (see Cairney, 2016). Whether better communication and more extensive use of knowledge brokers will suffice to bridge the gap, or whether more serious consideration of (and incentives for) sustained engagement and perhaps even the teaching of social sciences are needed to finally bring about more regular and effective use of research in policymaking is still debated. It is clear that the inclusion of impact in the REF2014 and REF2021 has provided the incentive for academics in the UK to pursue impact through their research even more actively than before. The following chapter will examine how the impact element of the REF2014 has been researched to date and will introduce the research design of this study.

19

THREE Researching impact

The criticism of impact as an element in the 2014REF focused on the ability to capture and assess the impact of academic work. The EBPM literature draws out the numerous barriers to the use of research in policymaking and therefore paints a bleak picture overall. Five years after the REF2014 a number of studies have analysed the process of the impact assessment as well as the ICS themselves, and yet confusion over what counts as impact in the REF context remains. This chapter will discuss the findings of existing research before introducing the research design of this study. This study aims to illuminate the nature of the underlying research as well as the nature and reach of the impact. The design is based on two comparisons of impact: across disciplines and over time, using a selection of ICS from the REF2014 submitted to the Sociology, Social Policy and Social Work and the Politics and International Relations sub‑panels together with interviews with UK Social Policy professors looking back at their influence and impact over their careers.

21

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Researching REF impact The studies analysing impact as part of the REF2014 broadly fall into three categories. The first set of studies have focused on the process of assessing and evaluating impact (Watermeyer and Hedgecoe, 2016; Derrick, 2018; Watermeyer and Chubbs, 2018). The second analyses the content of the ICS usually by academic discipline or subject (to be discussed later). The third group grapples with how to evaluate the scale of the impact claimed in the ICS (Smith, 2013; Morton 2015a; 2015b; Smith and Stewart, 2016). I will return to these arguments in Chapter five where I am developing and applying my own typology of impact. Starting with the first group, a number of ethnographic studies have explored the assessment process by the REF (sub-) panels through observing pilot exercises run by universities (Watermeyer and Hedgecoe, 2016) and by interviewing members of the REF (sub-)panels before and after the REF meetings (Derrick, 2018; Watermeyer and Chubb, 2018). The definition of impact provided by HEFCE was left deliberately broad, so that it could fit the multitude of disciplines and activities submitted as part of the new format (see Chapter two). This open definition left the panelist with the need to fill the meaning of the two main criteria of ‘reach’ and ‘significance’. Rather than coming up with their own agreed definition of impact, panel members focused on whether the ICS were able to demonstrate a ‘change’ or ‘benefit’ (Derrick, 2018). It was decided that the introduction of a policy was seen as sufficient, as requesting that an ICS was based on the outcomes of a new policy was regarded as unrealistic given the timescales of the REF guidelines. Furthermore, interpreting ICS based on public engagement was regarded as particularly difficult (Derrick, 2018). Given that panel members were not necessarily experts in the topic, they had to take the narratives presented in the ICS at face value. That, together with the knowledge that some

22

Researching impact

universities had employed professional writers, left some panelists taking the ICS with a combination of trust and a pinch of salt. In response to issues such as the difficulties of attribution, causality and time-lag, panelists developed a notion of ‘centrality’ of the research contribution. This was resolved through the use of the counterfactuals, that is, assessing what would have happened without the activities described in the ICS. However, despite (or even because of) these difficulties, the ICS received high scores: over 90 per cent of ICS were rated three- or four-star, more generous than the ratings for the environment or research outputs (Derrick, 2018). Watermeyer and Chubb sum up the much more positive framing of impact among academics now by saying that notwithstanding we see in these accounts the panelists’ efforts in adjudicating the impact of research in the REF being shaped by the fragility of self-concept, the inadequacy of criteria, the inconstant and inconsistent use of evidence; the absence of theory…there is much, we might argue, for the social science and arts and humanist to gain from impact, despite earlier concerns, and moreover and more worryingly, to game (Watermeyer and Chubb, 2018: 11, emphasis in original). Yet, the high scores from impact were partly political. Subpanels received a strong steer that the ICS should receive positive reviews, and so they were rated as three-star by default, precisely in order to convince academics of the value of the exercise (Derrick, 2018). The content of the ICS The content of the ICS has been analysed by several authors, each with a focus on their profession or discipline (Ross and Morrow, 2016 – leadership in HE; Syed and Davies, 2016 –

23

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

business schools; Kelly et  al, 2016 – nursing; Marcella et  al, 2016 – information research; Smith and Stewart, 2016 and Haux, 2018 – Social Policy; Dunlop, 2018, Hayton 2018, Selby 2018 and Dover and Goodman 2018 – Politics and International Relations), research groups (Biri et  al, 2014) or particular issues (Fordham and Noble, 2016 – end of life care). Others have focused on the role of the assessment process (Manville et al, 2015; Watermeyer and Hedgecoe, 2016), or institutional contexts and strategies (Wilkinson, 2017). Some of the key findings are the transferability of lessons from preparing ICS across disciplines (Wilkinson, 2017) as well as the diversity of the case studies even within disciplines and that includes the audiences reached (Wilkinson, 2017). Of the single discipline studies, the work by Dunlop (2018) is most relevant for this book (though, see also Biri et al, 2014, for a similar article for Natural Science departments). Based on her analysis of REF2014 ICS for the Politics and International Relations sub-panel, Dunlop (2018) develops a framework to analyse the submissions. It includes the characteristics of the individual/s submitted, the sub-disciplines, the methodologies employed, the audience, the geographical reach and the impact claims made. In the following chapter the most successful ICS from three sub-panels, including Politics and International Relations, will be analysed based on an expanded version of Dunlop’s framework. Based on her analysis Dunlop (2018) identifies six challenges for the discipline on the basis of the REF ICS: • that academic research has been shown to inform policy but not more, that is, not directly shaping policy; • the prominence of heroic impactors, that is, single individuals, which carries the risk of distorting research in favour of having impact; • that the first round took academics perhaps by surprise and that the second round of ICS submissions as part of the REF2021 should, therefore, provide a clearer picture of the

24

Researching impact

‘strength and depth of politics and international relations impact relationships’; (2018; 186) • the risk of being driven by policymakers whims and timetables, which are in itself incompatible with academic research; • that the range of methodologies were well represented but that only a few ICS were based on theoretical work; • that majority of the case studies submitted were uni‑disciplinary. Many of the points raised by Dunlop (2018) are echoed in interviews with Social Policy academics (Smith and Stewart, 2016), such as that research critical of the government was less likely to have an impact, that there was an overemphasis on instrumental and conceptual use of research, that there was a link between the rating of the research and the ICS, that there was an absence of an ethics or contextual statement, that there was an over-representation of national versus local impact, that there was a lack of diversity in terms of the ethnic composition (though less so with regard to gender), and that there was a reification of existing elites (Smith and Stewart, 2016). These aspects will be investigated when comparing the ICS across the sub-panels in Chapter four. Capturing and comparing impact across disciplines

Only a few studies have tried to capture and compare impact across disciplines (see King’s College London and Digital Science, 2015; see also research by the UK parliament itself: Kenny et  al, 2017; and POST, 2018). Bastow et  al (2014) examined the impact of social science and social scientists based on a wider range of data including publications, citations in policy documents and interviews and by comparing the findings to STEM subjects (Physical Sciences, Medicine, Engineering and Mathematics and CAD disciplines (Creative Arts including Design, Art, Film, Drama, Media, and Creative Writing and the Humanities). The concept of impact has been transferred

25

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

from the Natural Sciences, Medicine and Engineering (Bastow et al, 2014), where the transfer and transformation of academic ideas leads to new inventions and commercial gain. Thus, the authors argue that both the transfer and the transformation can be more clearly linked back to individuals/individual research groups and the gain can be more easily quantified in STEM subjects compared to the social sciences (Bastow et al, 2014 – see authors for a fuller discussion of the differences between the social and the natural sciences). This is supported by Khazragui and Hudson (2014) whose attempt to work out the ‘sum of all the net benefits attributable to the research converted into monetary terms discounted over time and space’ of ICS (2014:55) is thwarted by the difficulty of translating the multiple and diverse gains into monetary terms. A number of studies have tried to establish the impact of research outside academia using different methodologies such as the (largely quantitative) sophisticated frameworks called Payback, SIAMPI and Altmetrics (for example, Donovan, 2011; Martin, 2011; Kwok, 2013; Bornmann and Marx, 2014). Many of these approaches have been developed as alternatives to peer review, which is considered as reproducing existing bias and is also time-consuming and thus expensive (see also McKay, 2012). Discussing these alternative approaches is beyond the remit of this book. However, it is worth noting that all the different approaches point to the difficulties of definition, scale and attribution of impact regardless of measurement techniques. With regards to the three main disciplines represented in the sub-panels for the analysis here, a number of points can be made: The articles in the special edition on impact in British politics (see Hayton, 2018; Dunlop, 2018; Selby, 2018; and Dover and Goodman, 2018) bring out the different aspects of impact in terms of critical scholarship, the role of theory, as well as the different challenges for research in the field of International Relations and (British) Politics. ‘British politics is an impactful, engaged and relevant area of political science in the UK’ and that the

26

Researching impact

inclusion of impact can act as a rejuvenator for the disciplines, which have suffered from the internationalization of the subject (Hayton, 2018; 371). The argument is made for more regular and formalized engagement in the intelligence community (Dover and Goodman, 2018), though it should continue to be critical (Selby, 2018) and able to take the long view even if some/much of the dialogue is now taking place on Twitter for some academics (Hayton, 2018). Social Policy was part of the REF pilot (see Chapter two) and was regarded to have performed well then. This is in contrast to the discussion of changes to Social Policy as a discipline that has been carried out elsewhere, which has highlighted the move away from an applied discipline focused very much on Westminster towards a discipline that reflects the greater complexity and intractability of social issues as well as the different levels of governance (Donnison, 2016; Glennester, 2009; Page, 2010; Powell, 2006; Wilding, 2009). However, the focus of Social Policy scholars on making a difference to people’s lives through better policies comes out strongly in the interviews included in this book. Halsey in his comprehensive study of Sociology sums up the discipline as ‘at once a critic of the established social system and a potential social accounting instrument’ (2004; 144), that has seen an increase in both quantitative and qualitative work and one that will probably always be marked by internal disputes. Bastow et  al (2014) compared the visibility of the different social sciences measured in the links of academics with different policy-actors such as government departments but also different voluntary sector groups and independent research bodies. On all measures Social Policy came out far ahead of Sociology, with Political Science and International Relations bringing up the rear. When all measures are combined, Social Policy is the most visible of the Social Sciences, with Sociology and Politics and International Relations further behind (Bastow et al, 2014; 158). These levels of visibility and connectedness go some way to explaining the differences in rankings of the ICS submitted to the three sub-panels.

27

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Analysing ICS across Sociology, Politics and International Relations and Social Policy and Social Work As mentioned earlier, the majority of studies analysing the ICS have been focused on single disciplines or topics. In contrast, the comparison presented here aims to draw out the similarities and differences within the social sciences. The comparison of ICS in the book is based on rich and publicly available data of the ICS from the REF2014 from three sub-panels. In the REF2014 work was submitted to 34 subject-based Units of Assessment, which were grouped into four main panels. Panel members consisted of senior academics, international members and research users. The latter could be representatives from industry or the voluntary sector, for example, and their involvement focused on the assessment of the ICS (see Appendix, Table A2, for a list of members of the relevant panels for the REF2014). The three sub-panels included in the analysis are all from Panel C and include Politics and International Relations (UoA 21), Sociology (UoA 22) and Social Policy and Social Work (UoA 23). Between them, they cover a broader range of disciplines: Sociology, Politics, International Relations, Social Policy, Social Work, and Criminology. These disciplines are relatively similar in terms of their theoretical and epistemological approaches and there is considerable overlap in terms of the topics covered between the three sub-panels and, by extension, disciplines. Possible further extensions would have been Economics or Law but that would have gone beyond the remit of this book. The results of the ICS made up 20 per cent of the overall weighting. Each submission from a university to a sub-panel had to include a minimum of two case studies and one case study per ten FTE members of staff submitted (HEFCE, 2012). Case studies had to be submitted on a four-page template and based on research that was rated at least two-star (HEFCE, 2012 – see Chapter two). It is not possible to establish the rating of individual case studies for reasons of confidentiality. Therefore,

28

Researching impact

the departments with the highest proportion of four-star case studies have been selected from the three disciplines with the aim of selecting around ten universities with between two and six submissions for each REF panel (see Table 3.1). An alternative selection mechanism would be to take the proportion of fourstar ICS submissions per staff, that is, the grade point average. This would have led to a very similar selection of universities and therefore case studies. However, given that the focus in this study is on the relative excellence of case studies rather than of departments, the first approach is preferred. The analysis of the case studies cannot claim to be representative of the disciplines included in the analysis. Even including all ICS would not support such a claim, as the selection is based on a number of factors such as staff numbers and strategic decisions of their likely rating by submitting schools (see Dunlop, 2018). Finally, like other scholars such as Dunlop (2018) and Smith and Stewart (2016), the documentation of impact, which is not publicly available, will not be included in this analysis. Table 3.1 sets out the details of the universities whose ICS submissions have been included in the analysis for the three subpanels. The aim was to get a comparable number of case studies from the three sub-panels. This led to different cut-off points across the three sub-panels. For a comparison of the impact ranking of the universities included with their overall ranking in the REF, please refer to Table A1 in the Appendix. Overall, 37 case studies were selected from Social Policy and Social Work, 30 from Sociology and 41 from Politics and International Relations. The cut offs were based on the proportion of four-star case studies within the overall submission (see below) as well as the relative number of submissions to the panel with Politics and International Relations and Social Policy and Social Work both receiving substantially more ICS (166 and 187 respectively) than Sociology (97) (HEFCE, 2015b). When comparing the ratings across the three sub-panels in Table  3.1, a couple of things stand out. The first is that the

29

Table 3.1: Number and ranking of impact case studies included in the sample by sub-panel Social Policy and Social Work

Sociology

% of ICS rated four-star

Number of ICS

% of ICS rated four-star

Number of ICS

% of ICS rated four-star

LSE

 6

100

York

 3

100

Oxford

 9

87

York

 4

Oxford

 4

100

Cardiff

 4

100

Lancaster

 3

 80

Essex

 4

80

 80

Sheffield

 3

73

UCL

 2

100

Manchester

 6

 67

Keele

 2

70

Kent

 6

 93

Loughborough

 2

 90

Bristol

 3

 63

Leeds

 3

67

Roehampton

 2

 60

Kings

 5

64

Bath

 5

 84

Open University

 5

 60

LSE

 7

61

Dundee UEA

 2

 80

Oxford

 4

 60

Bradford

 2

60

 2

 80

*

*

*

York

 3

60

Glasgow Caledonian

 2

 80

*

*

*

Strathclyde

 3

60

South Wales

 2

 80

*

*

*

*

*

*

Total number of case studies

37

Cut off 80

Total number of case studies

30

Cut off 60

Total number of case studies

41

Cut off 60

University

30

Source: Times Higher Education, 2014

University

University

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Number of ICS

Politics and International Relations

Researching impact

impact ratings in the Social Policy and Social Work sub-panel are substantially higher than those of the other panels. The higher scores may be a reflection of disciplinary orientation, that is, that for many Social Policy scholars, influencing the way policy is designed and implemented has always been one of the core aims of Social Policy scholars (Page, 2010, though see also Holmwood and McKay, 2015). Therefore, the cut-off point for the Social Policy and Social Work submissions was 80 per cent whereas it went down to 60 per cent for the other two sub-panels. Second, as Table A2 in the Appendix demonstrates, the overall impact score is frequently but not always mirrored in a very good overall ranking, for example, as in the cases of the LSE, Oxford, York, Kent, Essex, Sheffield, Cardiff and Lancaster. Obvious exceptions to this are South Wales, Keele, Leeds, Bradford and Roehampton who were ranked in the twenties or thirties in terms of their overall REF performance in 2014 but scored (very well) for impact. However, overall there is a clear dominance of Russel Group universities among the universities with the highest proportion of four-star impact case studies. Third, the size of the submissions differs substantially with a few large players (Oxford submitted nine ICS, and the LSE seven in Politics and International Relations; Kent and LSE submitted six ICS to the Social Policy and Social Work panels, and Manchester submitted six and the Open University five to the Sociology sub-panels), but a majority of universities who have submitted only two ICS to their respective panels. Taking the long view: interviewing Social Policy professors The analysis of the ICS will be contextualised by interviews with Social Policy professors who are able to look back over 40 years and can therefore place their own impact, as well as that of their discipline, in the context of changes to the welfare state itself, government ideologies towards the welfare state, and the

31

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

development of and their role in, international organisations. The focus on Social Policy academics only for this part of the book is due to Social Policy being an applied discipline where influencing the way policy is designed and implemented has always been one of the core aims (Page, 2010). Furthermore, Social Policy scholars in the ‘second generation’ have been instrumental in exporting Social Policy as an academic discipline across the world. Both the welfare state in the UK and the academic discipline of analysing it are considered frontrunners in many parts of the world (see Powell, 2010). The term ‘second generation’ is used here to contrast this group of academics with the first generation of Social Policy academics such as Richard Titmuss, Brian Abel-Smith and Peter Townsend. Richard Titmuss is often seen as the founding figure in Social Policy both in terms of its intellectual and moral focus but also in terms of becoming the first Professor of Social Policy. The work and influence of the first generation of Social Policy scholars such as Titmuss, Abel-Smith and Townsend is the subject of many books and articles (for example, on Titmuss’ life and work see Bulmer et  al, 1989; Dahrendorf, 1995; Alcock et al, 2001). However, the professional biographies and influence of the second generation of scholars who followed in the footsteps of Titmuss are only just being collated. This generation of scholars, mostly born in the 1940s, are now near official retirement age. The reasons for focusing on this generation of academics are as follows: many scholars of this second generation were instrumental in the broadening of how Social Policy was analysed, for example, including the impact of welfare provision on gender, race and class (Glendinning and Millar, 1992; Williams, 1989) and the internationalisation and globalisation of Social Policy (Glennester, 2009; Wilding, 2009). Second, these scholars would have been ‘the experts’ in their field at the time of the New Labour governments in the UK and thus the height of EBPM (Bochel and Duncan, 2007; Nutley et al, 2007). Finally, given the sample, it is possible to

32

Researching impact

look back over whole careers as opposed to arbitrarily chosen timepoints. The design of the qualitative part of the study is loosely based on Halsey’s (2004) study of a group of sociologists. Halsey (2004) carried out a survey of Sociology professors in the UK, asking them about their careers, their political orientations, their positioning within and view of the discipline, their mentors, and whether they would become a sociologist again. While adopting many of the topics, I have chosen to carry out qualitative interviews based on a life history approach with Social Policy academics. My topic guide for this study included questions on what had drawn the respondents to Social Policy and on their academic careers, but also on their impact and influences. Sixteen qualitative interviews with Social Policy professors from the second generation based in the UK have been carried out. The interviews were all recorded and transcribed. Most of the interviews took place in 2013 and the timing is significant in that the inclusion of impact as part of the REF had been announced but also the pilots had taken place and, certainly, some of the interviewees were aware that Social Policy had come out rather well in the pilot exercise to test the ICS format (see HEFCE, 2015a). Given the context, a deliberate distinction was made between impact as defined by the REF2014 and influence with the latter being open to interpretation by respondents. Conclusion Impact in the social sciences is being examined both across disciplines and over time and through two different forms of data collection. The ICS are narratives of successful impact within a prescribed format and a political context. The interviews with the academics allow for critical reflection over a longer period of time and can, therefore, include aspects such as the durability of impact across different governments. That said, many of the interviewees have also featured in the ICS, submitted by their

33

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

universities and some of those are included in this book. In fact, the decision to include ICS in the analysis of impact was based on the often negative response to the question of whether the Social Policy professors had achieved any impact over the course of their career. Therefore, the two ways of constructing impact have been deliberately set up as a contrast in this study. The following chapter will describe the nature of the research and impact of the ICS submitted to the Sociology, Social Policy and Social Work and Politics and International Relations sub-panels.

34

FOUR Dimension 2: The nature of the underlying research and characteristics of the researchers The second dimension of impact covered in this book is the characteristics of the research and researchers. Therefore, the aim of this chapter is to illuminate the nature of the underlying research and researchers in the ICS from three of the REF sub-panels included in this analysis, namely Sociology, Politics and International Relations and Social Policy and Social Work. Based on the work by Dunlop (2018), the first part of the chapter compares the nature of underlying research, its funding and publication outlets. In the second part, issues around the use of research in policymaking raised in Chapter two, such as technical versus normative contributions, and the timing and trustworthiness of research will be discussed. As outlined in the previous chapter, the analysis is based on the submissions with the highest proportions of four-star ICS for Sociology, Social Policy and Politics and International Relations. This includes 37 from Social Policy and Social Work, 41 submissions from Politics and International Relations, and

35

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

30 submissions from Sociology. I will use examples from the ICS throughout this chapter to illustrate key points. The underlying research Even though this book is about impact, the underlying research is the basis of and therefore at the heart of the ICS. Starting with the methodology, where available, the categories of qualitative methods, quantitative methods, mixed methods and evaluations and conceptual work were sufficient to capture the range of methodologies applied (see Table 4.1). No ICS used experiments or were based purely on theoretical work, and so these categories have been dropped. Comparing the underlying methodologies displayed different patterns and, perhaps, emphasis across the three subdisciplines In the Politics and International Relations sub-panel submissions, conceptual work makes up the largest number of case studies. For Sociology this place is reserved for quantitative studies, while the ICS submitted to the Social Policy and Social Work sub-panel are spread almost evenly across the different methods. However, the relatively high proportion of quantitative submissions – varying from a third for Social Policy and Social Table 4.1: Nature of the research methods (%) Politics and International Relations

Sociology

Social Policy and Social Work

Quantitative

 27

 46

 30

Qualitative

  1

 27

  1

Conceptual

 61

 23

 24

Mixed methods/evaluation

Nature or research

  0

  1

 35

Other

*

  0

  0

Total

100

100

100

N

 41

 30

 37

36

Dimension 2: The nature of the underlying research

Work and Politics and International Relations to almost half for Sociology – warrants further reflections. These proportions are unlikely to be a reflection of the disciplinary orientation, given the relative shortage of quantitative skills within the UK social sciences mentioned in Chapter two (British Academy, 2008), as much as the perceived robustness and usability of quantitative research in the context of EBPM (Parkhurst, 2017). When looking at the different methods applied, it quickly becomes clear that they cannot be so clearly distinguished and that many of the quantitative examples are, in fact, the quantitative operationalisations of new concepts. This is demonstrated by the examples in Box 4.1. The first example is based on data collection, namely a survey of incidence of bribery and corruption across a range of countries. The methodology is now being adapted in further countries and a contribution in itself, but the main impact was to provide policymakers with a much more fine-grained understanding of the phenomenon than before. The second example involves the construction of an index of human rights and democracy, thus taking inherently complex and qualitative concepts and turning them into quantitative measurements in order to facilitate comparison across countries and across time. Similarly, being able to capture and compare levels of deprivation in small locations is very valuable information for policymakers as it allows them to target and tailor interventions. Finally, the introduction of the concept of child well-being, as a more comprehensive way of capturing children’s experience than measuring different aspects of poverty, is original at a conceptual and empirical level, and the results, placing the UK toward the bottom of the rankings in the first wave of data collection, made headlines. Therefore, the contribution of academics here is the application of their subject and methodological expertise to introduce and then measure new or existing phenomena in a way that they can inform policymakers and the public and allow others to hold governments to account.

37

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Box 4.1: ICS based on quantitative research Influencing the work of Transparency International in monitoring and reporting on global corruption through an online survey University of Strathclyde – Politics and International Relations

Because corruption involves illegal activities of public officials, data about the scale and objects of bribery is not readily available. Without such evidence, policymakers are handicapped in identifying points for effective intervention. Rose’s survey research on post-Communist countries developed innovative measures to monitor the payment of bribes by citizens for public services. Transparency International (TI) is the world’s leading non-governmental organisation campaigning against corruption, and it has incorporated the survey methodology in its key research tool, the Global Corruption Barometer (GCB). From 2008 to 2013 Transparency International has conducted three major rounds of GCB surveys that interviewed upwards of 450,000 people in more than 110  countries on every continent. Results have been disseminated worldwide through the 90  national chapters of Transparency International. Rose’s expertise in sampling has also been used to improve value-for-money expenditure on GCB surveys in the many developing countries it covers.

Measuring human rights performance and assessing the quality of democracy University of Essex – Politics and International Relations

Essex research on developing quantitative indicators for assessing countries’ performance on human rights and democracy has informed the work of a number of international organisations. Professor Todd Landman’s research has been used by the UN Office of the High Commissioner for Human Rights in its provision of assessment frameworks and by the UN Development Programme in its work on democratic governance and sustainable development. Landman’s research on democracy underpins the main resources employed by the inter-governmental organisation, International Institute for Democracy and Electoral Assistance (IDEA), based in Sweden. These resources have been translated into four languages and are used to assess the quality of democracy throughout the world. He also provides training for IDEA’s 150 members of staff on the measurement and assessment of democratic performance.

38

Dimension 2: The nature of the underlying research

Targeting resources and interventions in deprived areas using small area level indices of deprivation in the UK and South Africa University of Oxford – Social Policy and Social Work

Since 1999, researchers at the Department of Social Policy and Intervention (DSPI) have undertaken a programme of research to produce small area level indices of deprivation, in the UK and South Africa. These indices are widely used in these nations by central and local government, regional bodies, civil society, academics and others, to analyse patterns of deprivation, to identify areas that would benefit from special initiatives or programmes, and as a tool to determine eligibility for specific funding, enabling governments and other bodies to target their resources more effectively. The methodology developed for England was subsequently used to produce indices for the other countries in the UK, as well as South Africa, and is increasingly being applied elsewhere in Africa and Asia.

The impact of research on child well-being University of York – Social Policy and Social Work

Jonathan Bradshaw and colleagues at York influenced UK and international measures of child poverty, child deprivation and child well-being. The multi-dimensional well-being measures have been adopted by UNICEF and the Organisation for Economic Co-operation and Development (OECD). The Office for National statistics (ONS) is now developing measures of child happiness based on our work. Our research highlighted how badly children in Britain were doing. Informed by this evidence, a government strategy was developed after 1999 and investment in children improved at least until 2010. As a result, child poverty and well-being improved in the UK. Our work contributed to moving the national and international discourse beyond a focus on income poverty.

There were overall fewer examples of ICS based on qualitative methodologies and therefore only two examples are shown in Box 4.2. Much of the impact of the research on sexualities and intimate relationships to change the understanding of the public and professional practice, through deepening the understanding of bisexuality and relationships, was achieved through collaborations with organisations such as Stonewall, Relate, Pink Therapy as well as the Equalities Commission, the Department for Education and the Department of Health.

39

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

The research team in the second example worked closely with a number of London Boroughs to adapt services to facilitate the inclusions of different groups of migrants. The resulting toolkit ‘Migrapass’ has been developed to apply across Europe.

Box 4.2: ICS based on qualitative research Changing policy, practice and public understandings of sexualities and intimate relationships in the UK Open University – Sociology

Through the production of policy and practice reports, public engagement events, provision of continuing professional development (CPD) and training for practitioners, and dialogue with key stakeholders in government, the research team on sexuality and intimacy in the OU has had a direct impact on policy and practice concerning intimate lives in the UK. In particular, they have effected change in policy and public understandings of both bisexuality and intimate relationships. Underpinning this work is a motivation to shape contemporary debates about our intimate lives to further social justice and improve quality of life.

Localising migration: improving service provision and interaction between local authorities and migrant communities in London University of Roehampton – Sociology

This case study focuses on the impact of ethnographic research on migration to the UK from South Asia and Eastern and Central Europe by a team of researchers in sociology at Roehampton. This research has enhanced knowledge of demographic change in British society and has had a significant impact on policy makers and providers in local, national and European policy communities. The research has contributed to changing attitudes, raising awareness, and shifts in policy and practice by local government in London with regards to migration and social inclusion to the capital. It has also contributed to capacity building activities and new policy tools to support social inclusion and labour market integration by new migrant communities in the UK and across the European Union.

The third methodological approach identified by Dunlop (2018) was that of conceptual contributions and this was particularly

40

Dimension 2: The nature of the underlying research

prominent in submissions to the Politics and International Relations sub-panel. The examples in Box  4.3 demonstrate the potential for change of conceptual contributions. The first example is the, now seminal, work by Ruth Levitas on social exclusion. On the curriculum in all Sociology A-level and university courses, it has not only informed data collection but, what is more important, it has transformed the understanding of key national and international policy actors such as the UN and the OECD. Similarly, the conceptual and practical policy work on improving the life of women and young people to make it free from violence has been taken up by a whole range of countries and international organisations.

Box 4.3: ICS based on conceptual research Transforming the definition and measurement of social exclusion and poverty worldwide University of Bristol – Sociology

Research by Professor Ruth Levitas (solely-authored and co-authored as indicated later) has transformed the definition and measurement of social exclusion and poverty in the UK and worldwide by national governments, the United Nations (UN), the Organisation for Economic Cooperation and Development (OECD) and the European Union (EU). It has also shaped the work of local actors in diverse contexts. It fed into the measurement of social exclusion in the 1999 Poverty and Social Exclusion (PSE) survey, which was distinguished by its incorporation of a social dimension into the measurement of social exclusion. Levitas took the lead role in developing the measurement of social exclusion in the 1999 PSE. Subsequent work involving Levitas on these issues was taken up by the UK Cabinet Office in 2006, resulting in the B-SEM (Bristol Social Exclusion Matrix) in 2007. The B-SEM forms the basis of the measurement of social exclusion in the 2012 PSE survey, the largest poverty survey ever undertaken in the UK. The impact of the 1999 PSE and the B-SEM has been global and profound since 2008 – nationally in the measurement of poverty and the use of direct indicators of material and social deprivation; and internationally in the measurement of both poverty and social exclusion. Public interest in the initial results of the 2012 PSE is indicative of the fact that the impact is continuing.

41

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

The ECOWAS Conflict Prevention Framework: gender and human development King’s College – Politics and International Relations

Research by the Conflict Security and Development Group at King’s had extensive conceptual, instrumental and capacity-building impact across two areas of policy – gender and human development – in the Economic Community of West African States (ECOWAS), the 15-member West African regional intergovernmental organisation (equivalent of the EU in Europe), with a population of over 300  million. The underpinning research on youth, gender, militancy and violence was funded by the Economic and Social Research Council (ESRC), the UK Department for International Development and Norwegian Foreign Ministry, and had direct impact on gender and human development policy in the Conflict Prevention Framework of the ECOWAS Commission (equivalent of the European Commission) and ECOWAS Council decisions. Impact includes the adoption of new approaches to youth and women to address conditions of violence, and the adoption and implementation of a strategic action plan to change practice regarding women, thereby generating ECOWAS efforts to enhance peace and security in the region, completely transforming the conceptualisation of youth and the situation of all women wishing to work in the field of peace and security.

The dissemination of findings through publications

Next the dissemination of findings through publications will be analysed. Using publications as an indicator for the quality of the underlying research is, at best, a shortcut, and at worst fundamentally flawed (see Table 4.2). However, the alternative – to read and assess all the publications associated with the case studies – would have gone beyond the remit of this book. I have included the publishing company of books as a measure of prestige. Nearly one in two ICS from Politics and International Relations include, at least, one single authored book with a prestigious publisher (Oxford University Press, Cambridge University Press or a comparable American publishing house) compared to one in ten for Sociology and Social Policy and Social Work. Even leaving the nature of the publisher aside,

42

Dimension 2: The nature of the underlying research

Table 4.2: Number and range of publications based on references to research (%) Type of publications (multiple answers possible) Books

Politics and International Relations

Sociology

Social Policy and Social Work

73

57

 32

Articles

95

93

100

Book chapters

17

23

 11

Research reports

34

23

 70

Books with OUP or CUP

44

10

  8

Number of case studies

41

30

 37

three-quarters of Politics and International Relations scholars are more likely to have published books compared to around half of Sociology academics and a third of academics involved in the case studies submitted to the Social Policy and Social Work panel. This chimes broadly with the analysis by Bastow et al (2014) of the relative importance of books versus articles in the three disciplines. The relatively low proportion of research reports for the Sociology submissions is potentially misleading, as a number of submissions included research reports in the impact evidence rather than the references to research, but did not also list them as references to research alongside articles, books and book chapters. The mechanics of the research funding and dissemination

When considering the mechanics of the research funding and dissemination, Table  4.3 indicates that a higher proportion of Sociology and Social Policy and Social Work research has received explicit research funding. The funding pattern is likely to be linked to the greater dominance of empirical work in the first two sub-panels in contrast to the conceptual work in the Politics and International Relations panel, which is less reliant

43

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

on obtaining external research funding. However, it is not clear whether all of the ICS that did not mention funding had indeed not received any; this may well instead have been an omission. For many of the case studies, the research funding amounted to several million pounds from a range of funders over the period of the research. Directly commissioned studies (outside the area of health research) were frequently linked to small grants (for example, £20,000) but could result in direct impact. Whether commissioned research should be permissible as an ICS given that impact is to a certain extent built in, was discussed in the REF impact pilot (HEFCE, 2010; as well as within the subpanels, Derrick, 2018). There are some examples where the findings have been imported into policy almost wholesale and it is difficult not to think about research being used tactically in these circumstances. However, in the majority of the cases here, the commissions came part way through the research, that is, for follow-on work such as applications to particular contexts rather than the original research. The number of ICS involving commissioned research are relatively similar across the three Table 4.3: Research funding and set-up (%) Politics and International Relations

Sociology

Social Policy and Social Work

Research grants

61

83

78

Commissioned by government/organisation

29

43

57

No funding or not mentioned

17

 3

14

Research centres/groups

29

30

32

Total

41

30

37

Funding/independence (multiple answers possible)

Research centres

44

Dimension 2: The nature of the underlying research

sub-panels, which means that the practice of research being specifically commissioned is more prominent in the Sociology sub-panel given the lower overall numbers. Research centres

Finally, the relatively even proportion of research centres (see Table 4.3) across the three sub-panels hides a number of differences across and within the panels. First, for Politics and International Relations these are mainly research groupings within departments rather than research centres with full-time research staff. The latter were much more prominent in the Social Policy and Social Work submissions. However, even here it is worth noting that the vast majority of those research centres (which in any case are counted per case study and, therefore, effectively double counted at times) are concentrated in three universities, namely Kent, LSE and York, with a few additional submissions from centres at Bath and Loughborough. Who has impact? The discussion of the role of academics in relation to the social and political world, these days based around the notion of the ‘public intellectual’ has long focused on the position of intellectuals in society and what this meant in terms of their support and affiliation to the ruling class and political causes. In other words, are academics (supposed to be) working in ivory towers and largely self-referential and value-free circles, as so-called ‘traditional’ intellectuals, or are they (meant to be) ‘authentic’ intellectuals steeped in and contributing to the social and political world of their time (for overviews, see Kurzman and Owen, 2002 and Eyal and Buchholz, 2010)? There is, however, a danger of too much engagement leading to co-option and the loss of critical perspective (Kurzman and Owen, 2002; Eyal and Buchholz, 2010). Linked to the role of

45

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

academics is the notion of ‘distributed agency’ in the form of epistemic communities defined as ‘a network of professionals with recognised expertise and competence in [a] particular domain and an authoritative claim to policy-relevant knowledge within that domain for issue area’ (Haas, 1992: 3). They stand in contrast to the ‘lone scholar’ identified by Dunlop (2018) in her analysis of the REF case studies. Looking more closely at the characteristics of the ICS authors then, Table 4.4 sets out the staffing composition of the submissions in terms of individual versus team submissions, gender and seniority. The lone scholar group has been sub-divided into three groups: submissions were classified as single if the overall narrative was almost exclusively focused on one person; they were identified as main applicant with team if there was clearly a key individual who was leading the proposals; and as team submissions if that was not the case. In addition, the table identifies whether the case study mentioned research centres or groups. The numbers in Table 4.4 are based on a small and purposefully selected sample. However, I would argue that they are important because of the strategic selection and tell a story of what has been regarded as broadly the most successful ICS and we can therefore draw conclusions about how impact and, by extension, the role of academics and academic expertise has been portrayed and evaluated in the REF2014. As with subsequent examples, it seems that submissions to the Politics and International Relations sub-panel are at one end of a scale, and those to the Social Policy and Social Work sub-panel are on the other. While around two-thirds of the submissions from Politics and International Relations were based around a single academic, this applied to only one-third of the submissions from Social Policy and Social Work, that is, around half as many. Sociology was arguably closer to Social Policy and Social Work with around two-fifths of the submission being based around a single academic. The difference in composition follows through to the gender of the main/sole applicant with many more submissions from Social Policy and

46

Dimension 2: The nature of the underlying research

Table 4.4: Main characteristics of the applicant/s (%) Politics and International Relations Sociology

Social Policy and Social Work

Lone scholars? Single application

71

43

32

Main applicant with team

 5

27

 5

Team application

24

30

62

Male

76

43

16

Female

15

23

22

Mixed

10

33

54

Not clear

 0

 7

 8

Professor

63

63

32

Not professor/not clear

20

30

11

Mixed with no clear lead applicant

17

 7

54

Male/female professor ratio (either single or leading a team)

8:1

2:1

1:1

Total number of case studies

41

30

37

Gender 1

Seniority

1

Note: Where there was a lead applicant I have counted their gender and seniority, where it was clearly a team application of two applicants of equal status but a mix of gender I have gone for mixed.

Social Work being from a (usually mixed) team (around half) than from a single application (around one-sixth). For Politics and International Relations, three-quarters of the submissions are either sole submissions by male academics or led by a male applicant with only one-tenth being based on a (mixed) team. As before, Sociology is somewhere in between the other two sub-panels. Seniority shows a similar picture with the proportion of case studies authored or led by a professor being substantially higher for Politics and International Relations and much lower

47

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

for Social Policy and Social Work, with Sociology somewhere in between. It is worth noting that the ICS were often based on several decades of research, essentially the life’s work of the authors. While the forward-looking nature of the ICS template does sometimes include information about career progression, it is generally difficult to ascertain when much of the impact had been achieved relative to the career stage of the authors (see Smith, 2010 and Bastow et  al, 2014 in Chapter two, p  15). This is the case even when the submissions are based on an individual, but even more difficult when they are based on a team or centre. There are a small number of examples, where the original research that eventually led to impact was in the form of a PhD thesis (see the ICS from the University of South Wales in Box 5.4). Perhaps the clearest indicator that combines these three aspects is the ratio of male to female sole/lead applicants amongst those at the professorial level: this is 8:1 in Politics and International Relations, 2:1 in Sociology and 1:1 in Social Policy and Social Work (see Yarrow and Davies, 2018, for an analysis of gender ratio of 2:1 in the Business and Management Unit of Assessment sub-panel). In other words, there were eight times as many male professorial lead/main applicants in Politics and International Relations than female professors, compared to twice as many in Sociology, and an equal ratio in Social Policy and Social Work. This gender split will be a function of the proportion of male and female students at undergraduate level, with Social Policy and Social Work having very high proportions of female students. However, I would argue that the combination of the focus on the individual in Politics and International Relations (rather than the submission from a team) combined with the gender ratio also suggests a more traditional approach to academia. In contrast, Social Policy has historically been a discipline that has allowed academics to enter later in life on the basis of professional careers or late doctorates.

48

Dimension 2: The nature of the underlying research

Comparison of ICS by type of university

Concern over the inherent inequality of research assessments within as well as across universities has been voiced continuously in the UK, which is why a comparison of ICS by type of university has been included in this analysis. Criticism of some of the early incarnations included comments that the panel ‘in its implicit policies…had fallen prey to just all the different variants of home bias. English universities are favoured, old established universities are favoured, panellists’ universities are favoured and British journals are favoured, although marginally so. Large institutions are also favoured’ (Doyle and Arthur, 1992 in Bence and Oppenheim, 2005: 146). The discussion earlier has shown that the level of inequality in terms of who is credited with having achieved impact is skewed towards senior and male academics, but that the extent of this differs between the submissions to the three sub-panels. Table 4.5 draws out the differences in terms of the universities from which the most successful ICS have been submitted (see Table 4.1). Using a fairly standard division of Russell Group, post-1962 and post-1992 universities (for a critical account see McGettigan, 2013, for a short overview). What the numbers in Table 4.5 do not show is the ranking of the universities by type, which in almost every case brings the Russell group universities out at the top, followed by the universities of the 1960s; the post-1992 universities, Table 4.5: Impact case study submission by type of university Politics and International Relations

Sociology

Social Policy and Social Work

Russell Group

 6

5

 4

1960s

 4

2

 5

Post 1992

 0

1

 2

Total number of universities

10

8

11

Type of university

49

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

where they make it into the selection, come last. This picks up on another inequality, namely, the generosity of internal research funding for projects, which at many universities tends to be around the £5,000 mark but frequently ran into £300,000 at one university and therefore provided a distinct advantage over academics from other institutions. The need for research to ‘fit’ the policy problem In Chapter two the main barriers to the use of research in policymaking were timeliness, fit, trust/support and communication. Those critical of including impact in the measurement of academic excellence have also raised the issue of attribution and the lack of arbitration with regards to what constitutes positive change. These and other issues will be picked up in the remainder of this chapter. Focusing on the question of research ‘fitting’ the policy problem, Head talks about a continuum with ‘discrete, bounded’ problems that are ‘linked to a particular set of information and actors’ (2008: 4). These problems are likely to require a technical solution from experts in this field. However, at the other end of the continuum are ‘complex, inter-linked and cross-cutting’ problems, which therefore call for a diversity of networks and stakeholders to obtain the relevant expertise and buy-in and thus move from the technical to the relational approach. The latter is characterised by multiple evidence bases which ‘inform and influence rather than determine’ policy options (Head, 2008: 4). The examples in Box  4.4 show how academic work has accompanied and safeguarded technical changes and, quite clearly, lead to change. This change frequently takes the form of improving/introducing governance, for example, for new data availability, in order to make the process safer, fairer and more transparent. The first example is linked to the increasing availability and release of administrative data in itself and as being linkable to survey data. The potential of identifying individuals/individual companies

50

Dimension 2: The nature of the underlying research

becomes much higher once information from several sources is triangulated and furthermore, a number of the data sets available hold very detailed geographic or otherwise information that makes someone identifiable. A breach of confidentiality by researchers using the data is likely to lead to a reduction in access and could therefore harm not just the scientific community but also the public at large. Therefore, improvements to the dissemination practice, which includes training for researchers, play a major role in both safeguarding data and facilitating its use. The following two case studies are similarly referring to the governance and access in this case of new technologies as well as biological information. In both areas trust in the management of data is very important for future co-operation by the public.

Box 4.4: Accompanying technical change Impact on the statistical confidentiality practices of data stewardship organisations University of Manchester – Sociology

Research at the University of Manchester (UoM) has developed new approaches, methods and algorithms to improve the statistical confidentiality practices of data stewardship organisations (DSOs), such as the UK’s Office for National Statistics. The research and its products have had significant impacts on data dissemination practice, both in the UK and internationally, and have been adopted by national statistical agencies, government departments and private companies. The primary beneficiaries of this work are DSOs, who are able to both disseminate useful data products, and protect respondent confidentiality more effectively. Secondary beneficiaries are respondents, whose confidentiality is better protected, and the research community, as without ‘gold standard’ disclosure risk analysis, data holders can be overcautious.

Emerging biomedical technologies: shaping practices and influencing policy University of York – Sociology

Professor Andrew Webster’s sociological research on developments in biomedical science has been impactful in shaping regulatory practice and influencing policy in relation to biobanking, stem cell research and regenerative medicine. In particular, his research has been used to: change

51

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES donation procedures to the UK Biobank; influence regulatory decisions made by the UK Stem Cell Bank Steering Committee (UKSCBSC); contribute to regulatory practices associated with clinical trial design and adoption, and inform the UK government’s investment strategy in regenerative medicine.

Strengthening global regulatory capacity for nanomaterials risks LSE – Social Policy and Social Work

Dr Robert Falkner’s research into international risk regulation for emerging technologies underpins the work of the Nanotechnology Policy and Regulation programme at LSE. On the basis of this work, Dr Falkner was tasked by the European Commission to lead the first ever comparative study of nanotechnologies regulation in the EU and US. This research has stimulated policy debates in the UK and Europe on how to strengthen regulatory capacity in the field of nanotechnologies. The research has highlighted, in particular, the importance of improved transparency about nanomaterials in consumer goods and supply chains. This research finding has influenced the conclusions of the first UK parliamentary enquiry into nanotechnologies regulation and has informed a recent shift in global policy debates towards comprehensive and mandatory nanomaterials registers.

Dealing with a complex problem such as climate change

At the other extreme, the examples in Box 4.5 each address a very complex problem (climate change) that requires a coming together of experts from different disciplines and backgrounds. While ostensibly on the same topic, the two studies cover very different aspects, namely the link between climate change and migration in one and the link between attitudes, habits and consumption in the other. What the two ICS have in common is the extent to which the findings have been received by government departments. In addition, the first example is typical of the impact pathway of a number of case studies, where the initial dissemination of the research to government officials and ministers was followed by appointments to expert panels.

52

Dimension 2: The nature of the underlying research

Box 4.5: Contentious and/or interdisciplinary topics Reshaping the global policy agenda on environmental change and migration University of Sheffield – Politics and International Relations

Professor Andrew Geddes’ research on international migration has directly impacted upon the thinking of officials and the subsequent reshaping of policy at national and international levels concerning connections between environmental change and migration. Impact has occurred in several countries and at different governance levels. The result is that a previously deterministic policy debate about environmental change triggering mass flight is now based on a changed and far more sophisticated understanding of the evidence with different assumptions now informing policy development. Geddes was appointed in 2009 by the UK government’s Chief Scientific Advisor to be a member of the 6-member Lead Expert Group overseeing the ‘Foresight’ report ‘Migration and Global Environmental Change: Future Challenges and Opportunities’ (MGEC) for the UK Government Office for Science, published in 2011. The report and associated work has had major international reach and has informed policies and practices in UK government departments (DFID, DEFRA) and the agendas and operations of the European Union (especially the Commission), World Bank and within the UN system.

From attitudes to practices: new approaches to climate change policy University of Lancaster – Sociology

For the last two decades, sociologists at Lancaster have demonstrated the centrality of social organisation and practice for climate change policy. This case study focuses on the impact of Elizabeth Shove’s research in particular. Shove’s work challenges the prevailing emphasis on individual attitudes and behaviours and shows that the consumption of energy, water and other natural resources is an outcome of shared social practices. Through innovative forms of interaction and collaboration (‘working parties’; exhibitions and so on) Shove has inspired organisations such as World Wildlife Fund, the Environment Agency, DECC, DCLG, DEFRA, the Scottish Government and the International Energy Agency to take social practices seriously as topics of policy, planning and intervention. Individual behavioural models are no longer the only point of reference in policy design.

53

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Proving the effects/effectiveness of a policy Given the time constraints of the REF exercise , the sub-panel members agreed that impact could be claimed if a policy was introduced, and that researchers did not additionally have to demonstrate the effectiveness of the said policy (see Chapter two, p  22 and Derrick, 2018). The example in Box  4.6, namely the introduction of Universal Credit, throws this into stark relief. It is clear that the academic submitting the case study was one of the initiators of the change in policy and that the policy itself marks a departure from previous approaches. In that sense, it clearly falls into the category of policy innovation. Yet subsequent changes to the design and problems with the implementation mean that it would hardly be described as a success five years on. This raises serious questions about the timescale of impact, but also the requirement of documents, and making value judgements about impact being not simply change, but having made a positive contribution. This point will be discussed further in the following chapter.

Box 4.6: The introduction of a policy rather than its effects/effectiveness? Development of the ‘single working age benefit’ and impact on welfare reform University of York – Social Policy and Social Work

A major element of the Welfare Reform Act 2012 is the introduction of Universal Credit, which is widely recognised as the most radical reform of the UK social security system since the 1942 Beveridge Report. Universal Credit will be rolled out nationally from October 2013 and affect the lives of millions of working age people. Universal Credit recipients who are working will be the main beneficiaries receiving more than under the previous Tax Credit arrangements (which are being abolished). Increased payments to claimants will amount to over £2bn with 3.1 million households benefiting (according to the government’s formal Impact Assessment). The work of the Social Policy Research Unit (SPRU) at the University of York (particularly the contribution of Sainsbury and Weston) has had a demonstrable impact on the development of the ideas and policies of both previous Labour

54

Dimension 2: The nature of the underlying research administrations and the current Coalition government. Sainsbury’s seminal ideas on the single working age benefit can be seen as having laid the foundations for Universal Credit.

Dunlop (2018) has raised concerns over the large proportion of ICS that are based on a single discipline. This absence of interdisciplinary research can also be observed in the comparison here. In addition, to the seeming lack of collaboration with colleagues nationally, only a handful of case studies mentioned international collaborations. The latter is surprising given that there are no potential conflicts of interest or competition. As discussed in Chapter two, the set-up of the ICS in the REF2014 is based on submissions from individual units rather than research teams across institutions, as had been envisaged in the Australian predecessor. Given the more generally individualised nature of academic rewards, it is perhaps not surprising that a substantial number of ICS centre around one individual, although this varies between disciplines, as discussed earlier, yet there are few examples of cross-referencing and, therefore, the acknowledgement of epistemic communities. The examples in Box 4.7 highlight this problem as they refer to methodologies that were developed in collaboration in separate case studies. Therefore, claims of ownership and contribution are finely tuned, which then raises the question of scale of the impact or, as the panelist in Derrick (2018) remarked, they would have been asking the question of the counterfactual: what would have happened if the supposed contribution had not taken place? However, the ICS were submitted to separate panels, which may have helped their respective claims to impact. The second example is more straightforward as the research by the University of East Anglia has contributed to the reviews carried out by Munro and is explicitly referenced.

55

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Box 4.7: The issue of collaboration and attribution Example 1: Predicting election outcomes

Predicting UK election results for the media and the public to improve televised programming and inform the voting public University of Oxford – Sociology

Building on innovative statistical models developed at Oxford, initially conducted by Dr  Clive Payne and Prof David Firth, Dr  Stephen Fisher’s research facilitates quick generation of predictions for the share of seats won by each party from exit poll data. As a consequence, Fisher has provided the BBC with prediction and analysis of election results from 1997 to the present…The 2010 General Election is the focus of this case study. BBC, ITV, and Sky News placed such importance on forecasting the election that they unusually commissioned 18,000 interviews with voters at 130 polling sites. This must have cost well in excess of £100K, though the actual financial cost of the exercise is confidential. Based on these interviews, Fisher applied the methods developed through his research with Curtice and Kuha to produce a forecast as the polls closed at 10pm. As most of the data from exit polls arrived in the evening, this analysis had to be produced speedily, 15 minutes after the final batch of data arrived. After 10pm, the BBC deployed Fisher to provide results-based prediction and analysis.

Increased public and media awareness of voter behaviour at elections University of Strathclyde – Politics and International Relations

Based on his acknowledged research expertise in the areas of voting behaviour, electoral systems and survey methodology, John Curtice was widely consulted by the media before and after the 2010 UK general election to provide expert predictions on the likely outcome and to explain the results. By engaging with a variety of non-academic audiences, he informed public debate and understanding, and influenced pre-election planning by the UK Civil Service. He also contributed to the election night coverage by the three major UK broadcasters by accurately predicting the final result based on exit poll data. This informed much of the election night coverage, particularly on the BBC.

56

Dimension 2: The nature of the underlying research Example 2: Child protection

Child protection: improving practice LSE – Social Policy and Social Work

The Unit’s work has had a direct impact on the child protection system in England. In 2010, the Secretary of State for Education invited Eileen Munro to review the child protection system, giving her a wide remit enabling her to address systemic factors (such as the inspection framework, statutory guidance and performance management systems) as well as front line practice. All 15 recommendations of her final 2011 report have been accepted and are being implemented. Munro’s research has had significant reach: she has given evidence to two state government reviews of child protection in Australia; and in Queensland a charity is running a campaign to persuade the state government to learn from her work.

Preventing child death from maltreatment University of East Anglia – Social Policy and Social Work

Since 2006 the University of East Anglia (UEA) has led a series of Government commissioned studies of all Serious Case Reviews of child death and serious injury in England. This work has provided the largest national database of analyses of child deaths and serious injury where abuse or neglect are known or suspected. Since 2008, the findings have informed public understanding, practitioner thinking, multi-agency child protection practice, policy and law – in the UK, and internationally. Both key child protection policy and practice reviews commissioned by the UK Government 2008–13, the Laming report (2009) and the Munro Review of Child Protection (2011), drew on this research.

Finally, almost as an addendum and, yet, one that is critical to much of UK social sciences is the case study of the British Election Study (BES) (and other large scale survey projects included as impact case studies not included in this analysis such as Understanding Society). The ICS rests almost exclusively on the use of the BES and therefore on the survey itself. The BES will have featured in a number of the ICS included in this book and elsewhere and has spawned a large number of studies and publications as a vehicle for research.

57

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Box 4.8: Infrastructure The British Election Study: changing the survey sector, informing parliamentary debate and public opinion, and creating a commercial spin-out University of Essex – Politics and International Relations

Research conducted at Essex as part of the British Election Study (BES) successfully demonstrated the validity of internet-polling methods. This research has been integral to the development of YouGov and the growth of the online-survey sector in the UK. In addition, the BES has: informed public opinion about the likely consequences of introducing Alternative Vote in UK elections; led to the creation of a spin-out partnership, BPIX, which conducts polls for major national newspapers; informed parliamentary debates about the causes of non-voting; and served as a template for national election studies in other European countries.

Finally, Derrick (2018) also highlighted that panel members found it more difficult to evaluate public engagement. Claims to have changed public perception are made in a number of case studies, often focusing on how topics are portrayed in the media, for example, changing the awareness of broadcasters of the context and implications of devolution and therefore informing subsequent broadcasting. A rather innovative example of public engagement is the collaboration between Tim Newburn and his colleagues at the LSE and the Guardian newspaper. The latter approached Tim Newburn with the proposal of a collaborative study of the 2011 riots in England with the aim to inform public understanding. As a result of the collaboration, the majority of the findings were published as Long Reads in the Guardian newspaper.

Box 4.9: Public engagement ‘Reading the riots’ and increasing public understanding LSE – Social Policy and Social Work

The Unit’s Criminal Justice group has carried out a significant body of research relating to youth disaffection, anti-social behaviour and policing. This led to Professor Tim Newburn being approached by the Guardian to

58

Dimension 2: The nature of the underlying research establish a joint research project following the 2011 riots in England. The ensuing research achieved very wide reach via conventional print and other media, informing public understanding of the riots and challenging conventional wisdom about their causes. A wide range of public figures reacted to the research and the Home Secretary’s response included the announcement of a formal review of police ‘stop and search’ practice. This was published in July 2013, and in a parliamentary statement the Home Secretary said she anticipated significant reform of the use of these powers.

Summary and conclusions This chapter has compared the nature of impact by using the most highly ranked ICS submissions of the three sub-panels based on an adaptation of the analysis by Dunlop (2018). It has examined the underlying nature of the research, the composition of the case study authors and the publications, as well as some of the issues with assessing impact raised in the literature. Furthermore, it has done so by comparing case studies submitted to three sub-panels: Sociology, Politics and International Relations and Social Policy and Social Work. A number of patterns can be observed across the three sub-panels. Research featuring in the submissions to the Politics and International Relations sub-panel was more likely to be of conceptual rather than empirical nature than the ICS submitted to the other panel. This links to the relatively higher proportion of ICS, which had not received or not mentioned any funding. It perhaps goes some way to explain the seemingly more traditional set of characteristics of ICS authors in this discipline, that is, a higher proportion of submissions based on individuals rather than collaborations and on that individual being male and in a senior position. Regarding some of the issues raised in Chapter two around fit, attribution and timing of ICS, the ICS examples illustrate the range of contributions: from accompanying technical changes often linked to secure and transparent storage and access to data; to documents of contemporary events such as the riots;

59

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

as well as contributing to tackling complex problems such as climate change. While this chapter has focused on the nature of the underlying research, the following chapter will explore the nature and reach of impact.

60

FIVE Dimension 3: The nature and reach of impact

In this chapter, I want to return to the question of the nature and reach of impact, the third dimension of impact covered in this book. The focus in the sub-panels included in this book is on changing policy, discourse or practice. Although there are some collaborations with the private sector, in the main the intended change is taking place at the policymaking and practitioner level (see also Bastow et  al, 2014). The previous chapter compared the ICS based on a number of indicators, and also highlighted particular themes such as technical contributions, big, interdisciplinary problems such as climate change, issues of timing and attribution being accommodated in the case studies. The list is not exhaustive, as I did not examine all the case studies submitted to the three sub-panels. The similarities and differences between the submissions are likely to reflect nuances of disciplinary differences that are missed in comparisons of a larger number of disciplines such as in the work by Bastow et al (2014) and King’s College and Digital Sciences (2015). The second step is to bring together the different analytical lenses and develop a comprehensive typology of impact.

61

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

While the academic community is familiar with peer review of publications, evaluating impact of academic work on this scale is still in its infancy and only a few studies exist that have tried to develop a typology and ranking of impact (these are Morton, 2015a; 2015b; Smith and Stewart, 2016). Based on interviews with Social Policy academics regarding their view of the inclusion of impact in the REF2014 and a brief analysis of the highest and lowest ranked ICS submitted to the Social Policy and Social Work panel, Smith and Stewart (2016) have developed an impact ladder that places the highest importance on ideas in terms of reach and significance. Therefore, at the top of the impact ladder sits influencing the overall framework or paradigm of how an issue is perceived. This is followed by contributing to establishing an issue, influencing how a particular aspect of an issue is perceived, assisting policymakers in making incremental changes, and, finally, providing support for existing policies, including commissioned research (Smith and Stewart, 2016: 118, Figure 1). The authors further argue that the degree and significance of impact moves in the opposite direction to the ability to demonstrate said impact. For example, a policy report commissioned by a policy source is at the bottom of the ladder in terms of the degree and significance of the impact but at the top in terms of the ability to demonstrate impact. Conversely, changing the overall paradigm in which policymakers think about an issue can only be demonstrated through the use of different language by policymakers (Smith and Stewart, 2016). Research uptake, research use and impact Morton (2015a) has developed an alternative framework to capture impact, called the ‘Research Contributions Framework’ (CAF) with the aim of better capturing timing, attribution and context (Morton, 2015a; 2015b). The CAF distinguishes between research uptake, research use and impact. Research uptake is defined as when ‘research users have engaged with the research:

62

Dimension 3: The nature and reach of impact

they have read a briefing, attended a conference…or engaged in some other activity, which means they know the research exists’ (Morton, 2015a: 406). Research use, in contrast, is defined as when ‘research users act upon the research, discuss it, pass it on to others, adapt it to context, present findings, use it to inform policy, or practice developments’ (Morton, 2015a: 406). Finally, research impact ‘changes awareness, knowledge and understanding, ideas, attitudes and perceptions, and policy and practice as a result of the research’ (Morton, 2015a: 406). The CAF was developed on the basis of an in-depth analysis of the impact of a small number of research programmes (Morton, 2015b). Both the impact ladder by Smith and Stewart (2016) and CAF by Morton (2015a; 2015b) engage with the nature and demonstrability of impact perhaps more than how the impact has been achieved in terms of the type of underlying research and both suggest a hierarchy of impact. Based on the analysis of a selection of Social Policy and Social Work ICS, Haux (2018) developed the following four types of impact: policy innovation, policy direction, policy discourse and changing practice. This framework will be applied to the broader spectrum of case studies, which in turn required extending it. To some extent, the categories are hierarchical in that I would argue getting a new policy introduced is a bigger change than affecting the policy direction or policy options/discourse. The three typologies by Smith and Stewart (2016), Morton (2015a, b and Haux (2018) are quite similar in many respects in their aims and, therefore, their final shape. The key differences are the value placed on ideas (Smith and Stewart, 2016) rather than manifestations of change (as in Haux, 2018) and what counts as change, that is, whether this is the introduction of a policy, as in the case of Smith and Stewart (2016) and Haux (2018), or the effects the policy itself has had, as in Morton (2015a; 2015b). All three refer to how the impact can be evidenced, which is an important part of the debate. Haux (2018) also identifies different pathways associated with impact to the impact types.

63

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

The introduction of new policies

The first group, new policies, is perhaps the most obvious, and was only found among the submissions to the Social Policy and Social Work sub-panel. It refers to the introduction of a policy measure in its entirety. Examples that fall under this heading are the introduction of Universal Credit in the UK and antipoverty interventions in Sub-Saharan Africa. Achieving impact on this scale can be difficult in terms of the timing and policy openings, but also in being able to attribute sizable change to an individual or a group of academics. Box 5.1 includes two ICS from the REF2014 that are examples of policy innovation. They are based on interventions in countries other than the UK. It is clear that they were introduced with an evaluation alongside and therefore the ICS rests not so much on the policy itself and the role played by the researchers in bringing out this policy than it does on the change which the policy itself has brought to the lives of those targeted. This goes back to Morton’s (2015a; 2015b) typology, which includes the change brought about by the policies in the assessment of impact. Box 5.1: Examples of policy innovation Evidence-based model for child trauma recovery in war torn contexts University of Dundee – Social Policy and Social Work

Research by the Unit of Analysis (UoA) has transformed trauma recovery for children in situations of on-going violence. Hitherto, trauma recovery was provided post-conflict. Trauma recovery programme research developed at Dundee and led by Dr  Barron has directly improved psycho-trauma assessment and intervention for over 6000 children in Palestine. The UoA has led the development of a battery of screening measures and the delivery of culture-specific programmes into counsellor practice (N=200). These changes in psycho-trauma recovery have developed throughout Gaza (N=5000 children), across the West Bank (N=1000) and into other Middle East countries (Jordan and Egypt; N=200). The research has led to the delivery of trauma recovery programmes for maltreated children in 30% of Scotland’s Secure Estate (N=50).

64

Dimension 3: The nature and reach of impact

Sustainable livelihoods and well-being University of Bath – Social Policy and Social Work

In Bangladesh, 50 million people live in poverty and around 28 million live in extreme poverty. To date, development agencies have focused almost exclusively on the needs of the poor and ignored those of the extreme poor. Building on years of poverty research in Bangladesh, researchers at the University of Bath have played a key role designing and then developing a £65 million programme, which is the country’s first national scale initiative focusing exclusively on extreme poverty. Impacts from the programme include improving the livelihoods of one million extreme poor people; helping NGOs design innovative programmes for the extreme poor; and embedding the discourse around extreme poverty in the polity.

Affecting policy direction

A number of studies claimed to have provided evidence on different policy options to inform decision-making and policy direction. The distinction to the previous group is that the achieved change is smaller, it is an amendment of an existing or new policy rather than a whole new policy. This category contained some of the most interesting examples in my view, in that one case study claimed to have prevented a policy change that would have had detrimental effects, while another argued that it helped to maintain the status quo. The latter argument is particularly reliant on supporting statements as the claimed success is the absence of visible change. Influencing direction can happen at the design stage for a new policy or be proposed as an amendment for an existing policy, for example, through providing policy simulations of different options. The target group for this type of case study has mostly tended to be the UK government. The example in Box  5.2 highlights the differences between policy innovation and policy direction. In the example from the University of East Anglia, research prevented a change to an existing policy that would have left recipients worse off.

65

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Box 5.2 Examples of new policy directions Influencing reform of disability benefits for older people University of East Anglia – Social Policy and Social Work

A potential policy change concerning disability benefits for older people (received by 2.44  million over 65s in Britain), which would have been based on an incorrect premise, has been avoided, partly as a result of research carried out in the submitting Unit. We highlighted a flaw in the income measure in analyses used in Government to conclude that disability benefits go to older people without substantial financial needs. Measuring income appropriately, our research showed that recipients of these benefits in fact tend to be on low incomes. We were quoted extensively in a Health Select Committee report and elsewhere. The policy change has been abandoned.

Changing policy discourse

The third group contained case studies that claim to have changed the policy discourse in terms of the framing of a group or an issue as well as through having contributed different policy options to existing discourse. Examples in this category range vary in terms of the extent of the change in policy direction. At one end of the spectrum, a new group is now regularly consulted about policy changes, at the other end the language of the reporting about a group has changed. The majority of the examples fall somewhere in the middle in terms of having either costed policy options, developed new/improved quantitative measurements for a phenomenon (usually commissioned) or provided an independent analysis of a contentious event/ phenomenon. The audience for the change in policy discourse are key members of the policymaking community, that is, the government, MPs, civil servants, think tanks and the voluntary sector.

66

Dimension 3: The nature and reach of impact

Box 5.3 Examples of new policy discourse Improving illicit drug policy University of Kent – Social Policy and Social Work

This research has had impact on two linked areas of illicit drug policy. First, pioneering research on the effects of drug decriminalisation in Portugal has shifted the debate on this issue in the UK, US and elsewhere towards an acceptance that decriminalisation is a viable and not harmful approach. Second, research on alternatives to imprisonment for drug-dependent offenders has moved debate towards supporting the expansion of treatment for such offenders in the UK and US. These impacts are evidenced in the citation of the research by policymakers and NGOs (including the British Sentencing Advisory Panel; The All Party Parliamentary Group on Drugs; the Home Affairs Select Committee; UK NGOs, Release and Transform; the US Drug Policy Alliance and the United Nations Office on Drugs and Crime), demonstrating a significant influence on policymaking as well as public debate.

Putting basic income on the international policy agenda University of York – Sociology

Louise Haagh has played a major role in putting alternative welfare and employment policy options on the mainstream agenda. Haagh’s comparative empirical research on basic income (BI) provides support for an approach to welfare that gives citizens unconditional, universal economic entitlements and multiple opportunities, through education, work, and social care, to acquire the economic stability needed to help themselves. Haagh has promoted this approach through a portfolio of international engagement with policy makers, international organisations and NGOs: most notably, the Council of Europe’s ‘Rights of People Experiencing Poverty’ project and its major guide ‘Living in Dignity in the 21st Century: Poverty, Human Rights and Democracy’; and her work with the Government of Canada’s National Council for Welfare.

European human rights and strengthening policy to prevent genocide LSE – Politics and International Relations

Professor Karen E Smith’s research into European Union policy in the areas of human rights and the prevention of mass atrocities underpins the work of the European Foreign Policy Unit (EFPU). On the basis of this research, and as Director of the EFPU, Professor Smith has conducted a study of European Union human rights policies for the European Parliament and

67

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES served as Co-Chair of the Task Force on EU Prevention of Mass Atrocities. These activities have stimulated and informed policy debate within the EU and improved public understanding of the issues of human rights and mass atrocities across Europe.

Affecting the practice of professionals

The final set of case studies is directed at the practice of professionals, mostly social workers and the police. The change that supposedly has been affected is to a particular group of professionals rather than (part of) a policy or policy discourse. It contains examples of (commissioned) reviews of practice, changes to practice guidance through new research as well as the development of different toolkits. The target audience for this last set of case studies has been the professional bodies, though, at times, via Westminster (Haux, 2018). The examples in Box 5.4 show that research has transformed the practice of a whole range of actors such as the police and the military but also social workers and health professionals.

Box 5.4: Examples of changes in practice Enhancing policies to combat organised crime University of Oxford – Politics and International Relations

Federico Varese’s research has redirected thinking on the (mainly local) nature of organised crime. Varese has shown that a significant factor accounting for the spread of organised crime is the efforts of individuals to escape arrest in their home country. His research, as well as his role on the Strategic Review (2010), has made substantial contributions to the restructuring of the UK’s Serious Organised Crime Agency (SOCA). In Italy and Canada, he has helped to shape and inform policy debates within the Anti-Mafia Commission, and Public Safety Canada, respectively. His work has also been cited as being particularly relevant by prosecutors in Italy. More specifically, Varese’s use of quantitative methods to analyse the transcripts of wiretaps has inspired Europol to organise the training of police and prosecutors in these methods, and to alter their approach to better combat organised crime.

68

Dimension 3: The nature and reach of impact Postscript: Varese was acknowledged in a novel by John le Carré called Our Kind of Traitor for his advice.

Individualising responsibility in war: shaping military ethics in western militaries University of Oxford – Politics and International Relations

In the last decade, the traditional view of military ethics among practitioners and scholars has been challenged by revisionist approaches to Just War Theory. David Rodin’s work has played a significant part in the development of the revisionist school and in its subsequent impacts on Western militaries. His research on individual responsibility in war, the limitations of traditional justifications of war, and jus terminatio (the ethical norms applicable to the termination of war) has – as part of the revisionist school – opened up new approaches to military ethics and the way militaries understand their rights and responsibilities and those of others in war. Rodin’s research has had significant impact in re-shaping training on ethics for military personnel and senior civil servants in the US, UK and beyond, and is informing changes in the doctrine and practice of Western militaries.

Contribution to new methods of homicide investigation University of South Wales – Social Policy and Social Work

Brookman’s research has produced new insights into the nature and circumstances of homicide and homicide investigation. The first four pages of the Murder Investigation Manual, commonly viewed as the definitive guide on homicide investigation in Britain, are based on Brookman’s research on the characteristics of homicide. The directives of the International Association of Chiefs of Police based in the United States (US) devote one of their top ten directives to Brookman’s proposals on broadening outcome assessments. The Prince George’s County Police Department in the US is currently considering implementing Brookman’s proposals to include Family Liaison Officers as part of their process of homicide investigation.

Preventing violence against women: developing policy recommendations and best practice guidelines in the UK, Iraqi Kurdistan and India University of Roehampton – Sociology

This case study focuses on Aisha Gill’s ground-breaking research on violence against women (VAW) in the UK, Iraqi Kurdistan and India as part of the Crucible Centre for Human Rights Research. Gill’s research has had a direct impact on local, national and international policymaking and

69

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES professional practice, in particular, in relation to ‘honour’ based violence (HBV) and forced marriage (FM). This has underpinned her work as an academic commentator, with a strong media profile, her reports and policy briefings on VAW for UK and international public and third sector agencies, as well as an expert witness for the Crown Prosecution Service on HBV and FM cases.

New knowledge and data capture

Having applied the existing typology to the ICS from the other two sub-panels, it became clear that an additional category was needed. For the analysis of the case studies of the additional sub-panels, I have therefore added another impact category, namely, that of new knowledge and data capture. It encapsulates contributions that have established new ways of measuring a phenomenon or developed new indices that have since been adopted by international organisations. The latter was quite a common example of impact in the Politics and International Relations sub-panel and for further analysis of those submissions, it may be advisable to add a sub-category that combines a change in discourse and new data capture, as the two often went hand in hand in those case studies. This group is arguably quite straightforward though it is linked to the previous group. However, the ICS demonstrate the importance of good data capture and it is an area academics across the sub-panels have been involved in and have thus contributed to policy change (see Box 5.5).

Box 5.5: Examples of new data/knowledge capture Policies for better treatment for employees with disabilities and long-term health problems University of Cardiff – Sociology

This research into treatment for employees with disabilities provides one of the UK’s Equality Performance Indicators and recommendations from it

70

Dimension 3: The nature and reach of impact having been incorporated into UK legislation. Hundreds of organisations have used the research to promote better treatment for employees with disabilities; public-sector employers, including government departments, have relied on it to meet their statutory duties and it has helped the UK to fulfil its international treaty and convention obligations in respect of people with disabilities. This research has helped the Equality and Human Rights Commission (EHRC) to fulfil its statutory role to promote equality and human rights.

Impact on working time, working conditions and gender equality policy formation of the International Labour Organization and other international organisations with an employment policy remit University of Manchester – Sociology

Research on gender inequalities at the University of Manchester (UoM) has informed and shaped the development of employment policies advocated by key national and international bodies – such as the United Nations’ International Labour Organization (ILO), the European Commission (EC), Eurofound and the Equality and Human Rights Commission (EHRC) – in their role as advisors of national governments and regulators. Impacts are twofold. By advancing international comparative analyses of gender inequalities in employment and job quality, EU employment policy has been informed. By analysing gender inequality trends, alongside evolving national policies, the research has successfully steered key debates around both ‘working-time’ and ‘work–life balance’.

Working toward more accurate regional expenditure data and fairer regional funding formulae in the UK University of Oxford – Politics and International Relations

Research by Professor Iain McLean and his team has demonstrated that for more than 30  years, the process of distributing public expenditure to the regions of England, Scotland, Northern Ireland, and Wales was hampered by inadequate data and inappropriate policy targets. Since 2008, this work has had two principal impacts: (1)  it has informed a change in the Treasury’s methods of collecting and calculating regional expenditure data, information which is used to guide policy across all government departments; and (2)  it has contributed to the acceptance of needs-based regional funding as an appropriate policy target, and has laid the basis for a fundamental reform of the funding arrangements for Scotland and Wales.

71

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Table 5.1 brings together all the case studies and shows their distribution across the different impact types. Some clear differences between the submissions to the sub-panels emerge. The submissions to the Social Policy and Social Work sub-panel included in the analysis here are almost evenly distributed across four of the five impact types but did not include an example of new knowledge or data capture. For the submissions to the other sub-panels the picture is different but similar to each other. The ICS for neither sub-panel feature policy direction or policy intervention in a substantial number. Instead the emphasis is much more on changing discourse, practice and new knowledge and data capture. As with all good typologies, it has not been possible to fit all ICS neatly into the different types and, therefore, the emphasis is less on the fit of all the different case studies to the typology than its analytical dimensions. Table  5.2 brings together the type of impact with the underlying research. While there is likely to be some overlap between the types, a difference in terms of the underlying contribution can be determined between the different impact types. It could be argued that policy direction, practice and new knowledge are more likely to be based on technical Table 5.1: Type of impact (%) Politics and International Relations

Sociology

Social Policy and Social Work

New policy/intervention

*

*

 22

New policy direction

*

*

 24

New policy discourse

 39

 23

 19

New practice

 32

 35

 30

New knowledge/data capture

 22

 39

*

Total

100

100

100

N

 41

 30

 37

72

Dimension 3: The nature and reach of impact

expertise, while the contributions to new policy discourses and interventions are likely to include more normative elements. As can be seen from Table 5.2, changing the policy discourse as an ICS is much more prominent in submissions to the Politics and International Relations and Sociology sub-panels than the Social Policy and Social Work sub-panel. Instead, the submissions to the latter panel include more policy intervention or new policies than any of the other sub-panels. I would argue that this is both a reflection of one of the key disciplines in that sub-panel, Social Policy, which is focused on the analysis and improvement of policies (Powell, 2010). However, it also reflects a different interaction or pathway to impact and a different understanding of expertise, as will be explained later. The category ‘new policy discourse’ is quite broad across the disciplines. Particularly for the submissions to the Politics and International Relations sub-panel, it may have been useful to include another category that combined new discourse with influence on practice. Many of the case studies in that group Table 5.2: Linking nature of research and impact Type of impact

Methods

New policy/intervention

New policy design

New policy direction

Modelling options Assessing impact Evaluating pilots

New policy discourse

Development of new indicators Draw attention to particular issue or group Evaluating practice in other countries

New policy practice

Development of new toolkits Reviews of current practice New interventions

New knowledge/data capture

New surveys/survey technology New indicators New statistical models

73

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

had influence on how international organisations viewed a phenomenon and that could lead to a change in guidance. However, as many of these organisations, such as the UN, do not have a direct influence on countries and do not have any practitioners attached, I have included it in the category ‘policy discourse’. The nature of the influence on practice claimed in the ICS seems to be similar across the three sub-panels and reflects the close links academics have with the relevant professions such as police officers and social workers. Pathways to impact The more recent requirement for academics to demonstrate impact as part of the REF is an attempt to place it on a more formal footing. Unsurprisingly there has been an explosion of literature on how to have impact consisting of very practical and hands-on accounts (Reed, 2016; Denicolo, 2013) and the excellent contributions of the Impact in the Social Sciences Blog by the LSE, which includes a call for academics to get out of the impact game as well as summaries of existing research. Recently, Cairney and Kwiatkowski (2017; see also Cairney and Oliver, 2018) in one such summary argued that ‘evidence based action’ is limited by the complex policymaking environments that include multiple levels, the rules and norms of institutions, networks of trusts, and the socio-economic context in which policymakers are operating. For research to have impact, they recommend a three step communication strategy that consists of ‘understanding your audience, identifying the right time to exploit the “window of opportunity” and engaging with real world policymakers rather than waiting for a “rational” or orderly process to appear’ (Cairney and Oliver, 2018: 2). Lindblom’s (1959) suggestions for bringing about incremental policy changes are: to identify the key rules of the bounded rationality of the policy actors, to identify policy aims that are realistic and do not deviate drastically from the status quo; to

74

Dimension 3: The nature and reach of impact

limit the options presented to fit within that; and to include testing in the strategy, all indicate that much of this knowledge has been around for some time. Impact agents The range of impact agents mentioned in the case studies are set out in Table 5.3. The first observation is the spread across and within the sub-panels. Perhaps the only clear patterns are the relatively even focus of Politics and International Relations on international organisations, UK and non-UK governments. I have included the EU in the group of international organisations and the proportion is substantial for Politics and International Relations as can be seen in Table 5.3. The submissions of the Sociology panel are the most evenly spread as they include governments in different countries but also think tanks as well as business. In comparison, the agents for the Social Policy and Social Work group are mostly concentrated around UK governments and practitioners. The latter is partly due to the disciplines that are included in/submitting to the sub-panel, namely Social Work and Criminology. Social Work has an obvious body of professionals attached to it, and the case studies from Social Work tend to include recommendations for practice. In a similar vein, a number of the Criminology submissions include recommendations for the police service and include training or other manuals. The nature and distribution of the impact agents matters for a number of reasons. First, a focus on a single country government rather than a combined body such as the EU means that academics are more exposed to changes in the political orientation of the government of the day. Therefore, the timing of research and impact becomes much more important. However, given the different work and agency of the actors which are identified later, it also reflects a different orientation between the disciplines. Politics and International Relations tends to be more focused on advice and better capturing of

75

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Table 5.3: Impact agents and pathways (%) Agents and pathways (multiple answers possible)

Politics and International Relations

International organisations

 46

 17

 19

UK government (incl Local Authorities)

 41

 50

 57

Social Policy Sociology and Social Work

Non-UK government

 15

 20

 19

UK parliament and politics

  5

  7

  3

NGO/think tanks

 17

Media

  2

 13

  0

Business

  0

 10

  0

Practitioners

  0

 27

 30

Other

  0

 13

  3

Advisory role

 20

 50

 39

No advisory role

 80

 50

 61

Total

100

100

100

Number of case studies

 41

 30

 37

phenomena, such as a better human rights/gender index. This is very much the work of international organisations. The examples of impact pathways in Box 5.6 show sustained engagement with various impact agents such as national and international governments, NGOs and the media. The proportion of case studies that involved official advisory roles (Table 5.3) show an uneven distribution across the subpanels with the submissions to Sociology showing the highest proportion of advisory roles with half the case studies, followed by Social Policy and Social Work with two-fifths, and Politics and International Relations with one-fifth. However, the figures do not show that academics tend to be on multiple advisory boards, often chairing them. This refers to back to the societal standing of academics and their role within broader networks of

76

Dimension 3: The nature and reach of impact

actors involved in policymaking which is discussed in Chapters two and four. The ICS shown later have involved the authors in multiple appointed roles and partnerships with NGOs.

Box 5.6: Sustained and multi-faceted engagement as a pathway to impact Influencing law, policy and public discourse on the accommodation of Muslims in Britain University of Bristol – Sociology

Professor Tariq Modood – awarded an MBE for services to social sciences and ethnic relations in 2001 – is one of the UK’s most prominent analysts of and commentators on Muslims and multiculturalism. He has developed firm research-based arguments for why and how Muslims should be accommodated in Britain. The impact of his work since 2008 includes: the operation of legislation on religious discrimination and incitement to hatred; the operation of racial equality policies in higher education; the increasing prevalence of his view among Muslims and others that Britain’s distinctive form of moderate secularism should be seen as a valuable resource for the accommodation of Muslims; his participation in the creation and enhancement of mechanisms of Muslim participation in governance; awareness raising and agenda setting through his regular contributions to public events and media discussions (traditional broadcasting and social media); and the continuing adherence to multiculturalism, despite challenges to it, among Muslim and government actors, and their awareness of the importance of the macroparadigm level of analysis that he has developed. Professor Modood was one of ten members of the National Equality Panel, he is a commissioner and member of the Commission on Multi-Faith Britain, advisor to the chair of the Muslim Council of Britain, a founding member of Building the Bridge, a local project bringing together different groups in Bristol and he has been very active in disseminating his research through various media outlets.

Developing effective strategies to prevent mass atrocities through the Responsibility to Protect University of Oxford – Politics and International Relations

Professor Welsh’s research focuses on the responsibility of states and international organisations to protect populations from mass atrocities, and

77

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES argues that effective protection requires preventive action in addition to reactive responses to unfolding crimes. The research has directly informed the UN Secretary-General’s 2012 report on the Responsibility to Protect, which shapes how UN member-states interpret their responsibilities, and is contributing to expertise and capacity building in the wider policy community. In recognition of the importance and impact of her research, Professor Welsh was appointed Special Adviser to the United Nations Secretary-General on the Responsibility to Protect (R2P) in July 2013. Welsh started work on this research as a lecturer and the dissemination of her work involved meeting with the UN General Secretary, the UN Special Advisor on the Prevention of Genocide, the Foreign Secretary of Australia, the High Commissioner for National Minorities of the OSCE as well as members of the European and UK parliaments and culminated in her appointment as an UN special advisor on the Responsibility to Protect.

Shaping policy approaches to Thailand’s southern violence as a political conflict University of Leeds – Politics and International Relations

Research by Duncan McCargo at the University of Leeds has changed the way in which domestic and international policymakers, NGOs, the media and the Thai public have understood and engaged with the ongoing insurgency in southern Thailand. Since 2008, this award-winning, ESRCfunded work has played a key part in building consensus around the need for a political solution such as autonomy or decentralisation in the region. The research has supported peace initiatives, changed the implementation of security policy, and provided a road map for international donors seeking to help end the conflict. Professor McCargo collaborated with Dr Jitpiromsri, later Thai government negotiator, and worked with the Thai military, police, NGOs, the UN chief negotiator in Thailand and MPs.

Impact ‘beneficiaries’ Biri et al (2014) talk about impact ‘beneficiaries’ in their analysis of the ICS of the natural sciences at UCL. For example, impact beneficiaries of improved polling are MPs and political parties as well as the public. This group is different from impact agents

78

Dimension 3: The nature and reach of impact

set out in Table 5.3. Impact agents are individuals or groups with political power or access to power to make changes and are usually, therefore, directly targeted by academics seeking to have impact. Box 5.7 highlights the range of impact beneficiaries in the case studies across the disciplines, ranging from individuals with particular health conditions, those receiving care, and all children in a certain country. The ICS are often based on extensive engagement with users and their lobby groups and the resulting interventions and practice/policy changes have led to improved outcomes for these groups.

Box 5.7: A direct impact on people’s lives Shaping policy and practice for those with, or at risk of sickle cell and thalassaemia disorders University of York – Sociology

Karl Atkin participated in a research programme that explored ways of best supporting those with or at-risk of sickle cell and thalassaemia from the perspective of patients, families, practitioners and policy makers. The findings have had an accumulated impact on: care standards for a range of healthcare professional and national policy bodies; public outreach; ante-natal care and screening policy; and have informed the training and education of doctors and nurses on how best to communicate with those at risk of inherited blood disorders. Findings have also contributed to the evidence base on social care and education, including providing practical guidance to teachers.

Shaping ethical provision, care quality and design sensitivity in new health technologies University of Lancaster – Sociology

In a context of austerity and shrinking public provision, developed societies are turning to new technologies such as telecare for health/ social care provision, and biosensors to facilitate citizens’ active self-care. Maggie Mort, Celia Roberts and Adrian Mackenzie’s research explores the overlooked ethical and social aspects of this trend focusing on ageing, reproduction and genetics. Through innovative engagements with policy makers, industry, citizens, health experts and practitioners, we provide empirical intelligence about how remote care for older people living at

79

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES home (telecare) and providing users with bodily data (bio-sensing) work in practice. Because the views and experience of users and citizens underpin our research, our interventions confer much needed legitimacy on subsequent decisions about healthcare technologies and provision. Our recommendations are adopted in local authority service re-design and have shaped corporate decision-making about product development.

Reducing child anti-social behaviour through effective parenting interventions: international impact on policy, practitioners and families University of Oxford – Social Policy and Social Work

Frances Gardner and her team in Oxford have been studying antisocial behaviour in children for two decades. This programme of research has been instrumental in demonstrating that parenting programmes are effective in significantly reducing antisocial behaviour, thus encouraging uptake of these programmes by bodies that play a major role in forming UK central government policy relating to parenting and child behaviour, such as NICE and the (then) Department of Children, Schools and Families (DCSF). Given that lifetime costs of antisocial behaviour are so high, these interventions are likely to produce high return on investment, with cost– benefit analyses (for example, NICE; DCSF) suggesting that over £200,000 per child could be saved. The impact of Gardner’s studies has subsequently expanded beyond the UK, contributing to family intervention development in US trials, and to policy change by organisations such as WHO and the UN Office on Drugs and Crime (UNODC), and by policymakers in New Zealand, Malta, Slovenia, Estonia and South Africa.

Improving evidence-based policy and programming for AIDS-affected children in Sub‑Saharan Africa University of Oxford – Social Policy and Social Work

Since 2005, a pioneering set of Oxford University studies has actively informed the development of evidence-based policy, practice, and programming for AIDS-affected children in Sub-Saharan Africa (totaling an estimated 85  million children, orphaned by HIV/AIDS or living with AIDS-ill caregivers). Key impacts include new policies: on psychosocial support; on ‘young carers‘ of AIDS-sick parents as well as orphans; and on child abuse prevention for AIDS-affected families. These are based on Oxford findings that revealed major effects of parental AIDS on children’s psychological, educational and sexual health. Crucially, the research has also identified modifiable pathways of risk and resilience that have been used to guide interventions. As a result, studies are extensively cited in policy documents of the South African government, US President’s

80

Dimension 3: The nature and reach of impact Emergency Fund for AIDS Relief (PEPFAR-USAID), UNICEF and Save the Children, and have been used to train over 10,000 health and community staff and to develop programmes reaching millions of children throughout the region.

Improving outcomes of social care services: the impact of ASCOT University of Kent – Social Policy and Social Work

This research improved policymakers’ and practitioners’ understanding of well-being among residents of social care facilities by identifying the factors contributing to residents’ quality of life. This research resulted in the development of the Adult Social Care Outcomes Toolkit (ASCOT), whose use is rapidly increasing both in the UK and internationally. Evidence from beneficiaries including policymakers, experts and service practitioners, as well as interviews with service users, indicates that ASCOT captures aspects of well-being that are highly valued by service users and policymakers alike.

Conclusion The impact typology developed earlier further highlights the differences and overlaps between the disciplines included in this analysis. Submissions to the Social Policy and Social Work sub‑panel were the only ICS to include development of new policies but none were categorized under the heading of new data or knowledge capture. In contrast, the submissions to the other two sub-panels contained numerous examples of the latter and more under the heading of policy discourse. Whether this difference is a reflection of research within the disciplines or the results of perceptions of preferences by sub-panels for particular kinds of research is not clear. Analysis of the impact agents – that is, those able to affect change – highlights the much broader focus of the ICS submitted to the Politics and International Relations sub-panel than the other two. This is as one would expect, and it also makes academics in this discipline perhaps less reliant on favourable national governments than those working on issues relevant to the Social Policy and Social

81

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Work sub-panel. The use of knowledge brokers and advocates is most prominent in the Sociology ICS. Finally, the range of impact beneficiaries demonstrates very well the nature and scope of the impact that academic work can have.

82

SIX Dimension 4: Taking the long view: looking back over 40 years of Social Policy Impact was included in the REF2014 for the first time. However, this does not mean that academics have not been actively engaged in communicating and influencing the world around them before that. The final dimension of impact explored in this book is historical perspective via interviews with 16 Social Policy professors reflecting on the impact of their work over the course of their careers. These interviews provide a critical framing of impact by drawing attention to the time lags that go well beyond the impact timetable given in the REF, the often partial adoption of research, the use of knowledge brokers and, perhaps what is more important, a clear sense of being part of an epistemic community and a clear sense of ambition for the scale of impact and the nature of the relationship with policymakers. Through looking back over the course of their career, the interviewees are able to contextualise their influence and impact over time. This provides a foil for a critical review of the definition and implementation of impact assessment as part of

83

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

the REF2014. The academics were also able to reflect on the impact of their work over their professional lifecourse, rather than in an artificial set period. The interviews highlight the importance of the political environment, the composition of political actors, the developmental phase of the welfare state, the wider context of HE at particular points in time and, more specifically, the extent of individualisation within universities. The majority of these points will be applicable to the other social sciences as well, although the relevance of the development of the UK welfare state is probably particularly high amongst Social Policy compared with other disciplines. The interviewees were generally quite critical of the concept of impact. This is partly due to the timing of the interviews, which were carried out in 2013, just as the submissions for the REF2014 were finalised. As discussed in Chapter two, acceptance levels of impact as a measurement of academic work has risen substantially since the REF2014. However, perhaps more surprising and less directly linked to the REF, it seemed to me a tale of two halves with regards to views of their own impact over the course of their careers. Two halves in this instance refers to the fact that, for many, achievements of impact came early, often when working with their mentors and then tailed off later on. However, it also refers to two halves of the sample with one group being quite emphatic that they had very little impact while the other was able to point to a series of examples. In the following sections of the chapter, I will discuss the examples of impact given by the interviews and draw conclusions regarding its nature, pathways to impact, their academic role and policy context. However, equal weight will be given to the variety of reasons for why respondents were sceptical of the impact of their own work.

84

Dimension 4: Taking the long view: looking back over 40 years of Social Policy

The importance of political climate The different political climate is evident within the interviews and in comparison to the REF assessment period. The assessment period for the REF2014 was January 2008 to December 2013. The impact activity and impact should have taken place during that time, though it could be based on research published since January 1993 (HEFCE, 2011: 1). The REF2014 assessment period spanned two different governments in the UK: the outgoing Labour government, under Gordon Brown, and the incoming Coalition government of Conservatives and Liberal Democrats under David Cameron and Nick Clegg. It also included one of the biggest financial and economic crises in the UK, which not only diverted the attention of policymakers but also severely limited the spending power of governments due to the renewed attention on fiscal standing of countries. In 1997, the New Labour government had set out to base policy on evidence and came into power with a relatively well-defined and ambitious programme of welfare reform (see Commission for Social Justice, 1994). In tone, content and operationalisation, policymaking in terms of welfare was going to be different under New Labour than the previous Conservative governments (see Chapter three). The Labour government introduced a range of new policies and programmes to meets its overarching goals of ameliorating child poverty, to make work pay and so on. However, even before the financial crisis hit in 2008, public finances had become much tighter than at the start of the Labour years in government and, hence, academics needed to demonstrate what the famous marginal pound should be spent on (Lupton and Burchardt, 2016). The interviewees talked about occasions where they had set up pressure groups that are still in operation and effective now, or when their research had identified the needs of a group that had hitherto not been noticed. In the general climate of welfare state expansion, the examples of impact are, at times,

85

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

staggering. In terms of the timescales, the interviewees have talked about their experience as students in the 1960s all the way to the Coalition government. Therefore, they are spanning 40 years of welfare state history and it would go beyond the scope of this book to even try and sketch this out coherently. For authoritative accounts see Timmins (2001), Thane (1996), and Fraser (1992). For this group of interviewees, this meant that they often had a lot of influence in the early stages of their careers such as being involved in the founding of the Open University, founding and running key lobby groups such as the Disability Alliance and the local and national Child Poverty Action Groups, as well as making a successful case for the expansion of financial support and services to a greater number of groups such as people with disabilities and ageing carers. Many of these activities would have been carried out with their mentors and as part of a broader group of academics. The history of the welfare state in the UK, as well as the changing political views of successive UK governments, meant that, quite by chance, the academics in this generation spent their most productive years at a time when they would have been able to have impact, according to Bastow et al (2014), either fighting the dismantling of the welfare state by respective governments or finding other spheres of influence, with a number of academics getting much more involved in international organisations. Examples of impact One such academic felt that Europe provided a platform for engagement and contribution of policy ideas: But like I said, I think Europe is different. I mean one of the ways in which I feel I’ve had impact is…being invited by the European Social Platform to…and that was wonderful because they represent lots of different NGOs

86

Dimension 4: Taking the long view: looking back over 40 years of Social Policy

and grassroots organisations across Europe. And they have a direct…channel into the European Commission (EC) who sit and listen to them – they took care as one of their central themes a couple of years ago, asked me to go and give two sort of…afternoon long seminars with some of their key people from international organisations. Sat down, worked with some of my ideas, came up with ideas, sent them back to me. I acted as a sort of advisor, unpaid advisor…to them and then what they came out with which was their recommendations to the EC started off with a statement which was straight out of my – just paraphrasing all my work on political ethics of care, as to why care was central to our – our, you know, society, human flourishing, etc etc, and they came – they emerged with this statement which was that…‘we believe that the EC should recognise the fundamental right to receive and provide good quality care’. And I thought that was brilliant. Absolutely brilliant, you know? That was a process of interpretation of a, you know, my statements which have come out of research as to how policy might think, retranslating it into a principle to guide policy…And I – and I’m very proud of that. (Academic G) Another example of impact of changing policy direction, this time nationally, is research I’ve done has had an impact on government policy, even on Tory government policy. I did the first project in the UK on older work, on the attitude of employers to older workers in the mid-eighties and – no early nineties – and it made the government change the policy. They ran an age discriminatory policy and they were excluding older workers from public training schemes, and they changed that. (Academic C)

87

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Contributions to policy discourse were not regarded as impact per se by respondents but, nevertheless, as an important element of their work, as it meant that ideas were around to be picked up by governments and policymakers if and when. As one respondent argued take an example which is on the margins of academia, Help Aid International have been banging on quite correctly about universal social pensions as an idea. And you can now point to the establishment of at least policy debates about universal social pensions in a whole number of forums. So you can trace back if you like the spread of – the debate about policy idea in some ways. So it’s not just citing people’s scholarly work, it’s citing somehow or other the policy issues that they have been party to discussions about. So in other words, somebody may have totally failed to persuade the UK government to change its policies – completely failed but they have influenced the discourse about it. So Polly Toynbee is on the losing to nothing at the moment, but she’s very important because she captures a discussion like that. So it’s about capturing, it’s about framing the discourse and ways which then are sustained in some way? (Academic H) Finally, an example of data capture and knowledge that is typical of a combined policy idea and measurement impact is the introduction of measuring child well-being in addition to child poverty. This was a collaborative effort by an international group of scholars of whom one of the respondents was a key part: I think that’s one stream of research which I could claim some responsibility for. The other is the work, more recent work, we’ve been doing on child well-being in two kind of streams of that. One, I’ve been associated with the UNICEF Innocenti Centre for some time now and have

88

Dimension 4: Taking the long view: looking back over 40 years of Social Policy

been helping them with their international comparative studies of child well-being. And we published the first sort of social indicator based on an international comparative study of child well-being and that’s been influential in establishing the International Society for Child Indicators, which I just left the board of, and the journal in child indicators’ research. But we’ve also been doing a series of studies with the Children’s Society, mainly focused on subjective measures of child well-being, which has begun to influence the work of the Office of National Statistics who’ve had to respond to Cameron’s call for us to be interested in well-being as well as…happiness. And they didn’t have a clue how to approach the measurement of children’s happiness, so that work that we’ll be doing with Children’s Society was all that there was out there, really. And we are now collaborating with them in their very tentative efforts to measure children’s happiness and the Children’s Society has just actually yesterday, no today, published a new report on child well-being which we contributed to. And OECD has published reports on child well-being now and UNICEF is still very actively engaged in international comparisons of child well-being. (Academic A) The examples of impact discussed earlier are a selection that reflect the diversity of impact types but also the reach and connections of the work of these academics in terms of their international presence. In terms of pathways to impact, a number of now familiar themes were raised, such as the importance of forming alliances, using the language of politicians of the day, lengthy engagement with policymakers (ideally at the beginning of the policy-design process), and ongoing engagement with the broader epistemic community, which includes civil servants, politicians, other academics, think tanks and the voluntary sector. Thus, membership of advisory boards, and including

89

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

policymakers in the broader sense on international project boards was really important. Critical view of (own) impact It is perhaps more instructive, however, to explore the critical view that the interviewees had of their own impact and the current definition of ‘impact’ by this group. Here, the ability of the interviewees to look back over 40 years of engagement with policymaking both as participant as well as analytical observer provides an in-depth and long-term perspective that is not present in the ICS. Expectations of this generation of the level and kind of impact achievable

One of the reasons why there was a strong sense of scepticism about their own impact is that this generation had very high expectations of the reach and significance of impact that they would have. This expectation was partly shaped by the experience of their mentors, many of whom had close relationships with key ministers in the UK government in the 1950s and early 1960s. These mentors became a point of comparison for many of the academics of the second generation and shaped their expectations of what their relationship with policymakers would be like throughout their academic career and, related to that, the sort of influence that they would have. Yes, it [the LSE] was a stimulating place, because the first generation, in your terminology, were all there. They were all advising ministers. Brian Abel-Smith would say, well if you want to see me you’ve got to see me before 8.30 in the morning because at 8.30 I’m off to see Barbara Castle…David Donnison would talk about what’s been happening in the Department of Environment or whatever it was called then – Housing and Local Government

90

Dimension 4: Taking the long view: looking back over 40 years of Social Policy

I suppose, advising Crossman…You know, so it was a hugely stimulating place to be and you felt you were on the fringe of making policy although you weren’t really. But you were mixed up in that very heady environment really. So that’s the beginning of the story. (Academic B) The second key aspect is that the interviewees had impact goals that were hugely ambitious. For many scholars of this generation, one key goal was to abolish poverty, particularly child poverty. This level of optimism and belief in the power of government intervention is probably a sign of the time, but serves as a reminder that the overall goal for many academics and activists is far larger than the often relatively modest contributions outlined in the REF2014 ICS. However, at the time of the interviews in 2013, child poverty in the UK was rising again, and government consultations on how to measure child poverty had suggested moving back to individual rather than structural causes. Thus, any progress made under the previous Labour government – in terms of not just of policy goals and investment but also of framing – was being rapidly lost. This probably explains the bleakness of the assessment of one respondent: You could really say that the climate we’re in now is evidence of our complete failure. I mean, we’ve got a government who’s completely impervious to everything that our discipline has argued about for 70 years, totally impervious to evidence, holds values and beliefs which are completely at odds with ours. And is able to communicate them quite successfully. I think that’s the most shocking. (Academic A) The third reason for the scepticism of their own impact was a keen awareness of the changing fortunes of any influence, and the serendipity and selectivity of research use. The ability to look back over 30 or 40 years gave the interviewees an awareness

91

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

that policy victories were often ‘temporary’, as they could be reversed. Similarly, it meant that the ability to have impact depended on how much the research or recommendations fitted with the government’s existing thinking; when it did, though, ‘impact’ could come very suddenly: I was saying this in, you know, ’78, ’82, nobody took any notice. When you are saying something that’s in the spirit of what they want to do, I mean I got a letter which I was terribly chuffed with at the time, positive impact about how – in about ’98, ’99, got this thing from Blair ’cause we all worked on the speech that announced the abolition of the aspiration ending child poverty, and I got this letter from Blair saying, ‘I really enjoyed this, that ’cause I thought this was really interesting, this has really shaken my thinking on this.’ And of course it’s complete cobblers, I mean he was instinctively pro New Deal and, you know, and this was kind of perfect ’cause this was some sort of academic rationale about conditionality. So it was an impact in that way, it’s just it gets used I think. And I think that’s very true. (Academic E) Another example of the serendipitous use of ideas and the role of the broader policy context is: So, you know, in the ’70s and ’80s I contributed to a debate that was about getting rid of discretion against the means tests and, you know, all that was moving in the right direction. Well my views haven’t changed, I think the evidence hasn’t changed, but the climate has changed, you know. So when Ed Balls comes out yesterday and says, ‘We’re getting rid of heating allowances,’ you know, on the basis of income, not on the basis that there are other ways to do it if you want to save the money. But, you know, it’s just a sign of [universalism], it’s not got a good future

92

Dimension 4: Taking the long view: looking back over 40 years of Social Policy

I think. So it’s more about feeding into a broader climate if you like. (Academic C) Indeed, some respondents were able to point to ideas being picked up nearly 30 years after the initial publications: So going back to your ‘what influences policy?’, has something been written in the last four years during the period of the REF made any difference to policy? Almost certainly not. Will it in forty years’ time? Well possibly. If somebody was doing a REF in 1968 and seen my idiotic article on the graduate tax they would have just thrown it in the wastepaper basket, completely unrealistic. All the correspondence I got at the time saying, you’re mad, this is wicked and it only bore fruit, for good or ill, forty years later. REFs can’t capture that. (Academic B) Academics in this generation were well aware of the pitfalls of ideas being adopted and then adapted in the process until they are unrecognisable from their original intentions. One such idea was the concept of active ageing ‘but the way that’s applied in most countries and in the Lisbon criteria from 2000 and – from 2010, is working longer; it’s all it is, make people work longer, which is a travesty’ (Academic G). However, the same concept was introduced in line with the original intentions in a number of other countries. Critiquing the construction of impact in the REF A broader critique of the construction of impact in the REF was put as follows: the problem for the REF is twofold. One is whether impact is the right way to think about how knowledge works, but the other is, and it’s the political problem, who

93

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

does not want to produce useful knowledge, right? Don’t we all? Didn’t we all? Wasn’t that a – a demand about how critical work was going to change the academy? So, we’re always caught by the demand to have impact because of course, we think that we should be useful, effective. But the problem is that its specification as impact rather than influence, and then its operationalisation in such a narrowly economised, instrumentalised vocabulary is an example of what a lot of Janet’s (Newman) work talks about which is, you know, we are all in favour of producing useful knowledge and then it gets translated…into things that you think – There’s somebody in Janet’s book who says, ‘you have these words and they get taken up and they come back and bite you on the bottom’. (Academic I) It could be argued that the implementation of Universal Credit, which was an ICS in the REF2014, fits this description rather well. Furthermore, there are probably going to be several ICS in the next REF that will argue that they contributed to highlighting the numerous problems with Universal Credit and, therefore, helped to slow down the roll-out of this policy. The fourth reason that may explain the sense of not having had much impact among this generation could be that much of their most productive years, according to Bastow et al (2014), were spent preventing further cuts to the welfare state under successive Conservative governments in the 1980s and 1990s, and that this will often have seemed futile: I think quite a lot of the period of my life where I was most productive and possibly influential was during the Thatcher years, where together with other people we stopped things happening and you can’t demonstrate that. So if you look at the work that was published in the ’80s and early ’90s by the Joseph Rowntree Foundation, work done by John Hills and colleagues, and associated

94

Dimension 4: Taking the long view: looking back over 40 years of Social Policy

with the Barclay Commission, which demonstrated the welfare state was not as everybody imagined it to be, a kind of vehicle for redistributing resources from the rich to the poor, but a way of redistributing resources over one’s own lifetime and between generations. And our tax system wasn’t progressive, in fact it was regressive, all those kinds of pieces of evidence, which helped to oppose the neoliberal agenda. I mean, I think they were great achievements but they were negative achievements, in a way. And I think quite a lot of my work’s really been of that kind, fighting prejudices, rather than proposing radical changes; defending what we’ve got, rather than changing it. (Academic A)

Being part of a broader community Finally, a key reason for why academics were sceptical of the operationalisation of impact in the REF and their own influence was that they had a very clear sense being part of a broader epistemic community. They spoke of common purpose and strategies both among themselves as well as with policymakers. In fact, part of the difficulty of the project of trying to identify the impact of this generation of academics was the strong sense of collective identity and influence. This sense of collective identity had been shaped very substantially by the experience of having mentors and working collaboratively in their early years and more generally working alongside senior colleagues in the field. Time and again, a number of academic teachers are mentioned as mentors in terms of validating interests but more often in working with younger scholars. This could take the role of assistants and data collectors, but in many instances the young academics seemed to be encouraged to write their own papers or develop future projects or even jobs out of the collaboration, clearly operating as a step up the career ladder.

95

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Moreover, these mentors also taught and collaborated with the younger scholars about pathways to impact. It is almost as if being an academic was regarded as an apprenticeship and that it was the responsibility of mentors to assist the younger scholars to have opportunities to learn and practice those skills. Perhaps, due in part to the influence of mentors on the academics in the study, there was a strong sense of being a ‘purveyor of ideas’ and standing on the ‘shoulders of giants’ rather than acting as an individual. Or, as one interviewee phrased it: ‘it’s this problem, you don’t have concrete impact but you have relationships to debates’ (Academic D). Similarly, there was a clear awareness of the collaboration between academics, and the different roles that they and other individuals played in the joint endeavour of bringing about a particular policy change. On such example of the shared understanding is the account of the idea of abolishing child poverty: I think that was Robert Walker. I think Robert was the key player there. A number of people were invited to submit ideas for this speech. And it was the classic New Labour mixture of kind of opportunism and genuine new idea thinking, because I’ve absolutely no doubt that the commitment on child poverty was to sort of spite Brown who was going to say something a couple of weeks later, you know, it was that element to it. But I can’t claim any credit whatsoever for the pledge to abolish child poverty. I think Polly Toynbee suggested that and, I think, Ruth Lister did. But I think it was Robert originally…Robert’s much more involved than most of us, ’cause he was on the social security advisory committee at one time. (Academic E) The idea of individual attribution was regarded as rather problematic:

96

Dimension 4: Taking the long view: looking back over 40 years of Social Policy

Yes. You see I think that in a sense the word ‘impact’ is a metaphor, isn’t it? I think it’s quite, if I may say so, quite a male metaphor. You know, as I say, it is like shoot the gun and hit the target. I was trying to think of a metaphor that would work better, and the only one I had come up with was the idea of a street party. Where somebody has an idea, but it doesn’t [happen] unless lots of people contribute. At the end, you have the party. But in order to get there, somebody has to produce the tables and chairs, and lots of people have to produce food and, you know, plans have to be made. And, you know, games have to be organised. And somehow, having an impact in research I think is more like a street party than like firing a gun and hitting the target. (Academic J)

Academic identity In terms of their own identity as academics, something that stands out from the interviews is the level and variety of political engagement in the early years of the interviewees. Pretty much all of them were involved in local organisations, many of them setting up or working at local branches of the Child Poverty Action Group, and others with trade union involvement or setting up refuges or starting the Young Fabians. Often the engagement in organisations was shared with mentors; one respondent referred to it as the ‘application of the academic study into actual practice which again is the influence of Townsend, ’cause that was very much where he was’ (Academic  K). However, it is not only the activism that is striking but also how closely it is linked, for many of the interviewees, with the understanding of Social Policy. The level of engagement and understanding of more local politics has been carried on by many of them in their research or other engagements long into their career.

97

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

I’d always had a strong view that you had to make a difference in the real world and that wasn’t just an intellectual difference, that was actually going out there and doing something and doing the welfare rights. I was on tribunals for a while as well, sitting as a member of the tribunal making decisions, cor blimey that was difficult as well, so it got me interested in that. (Academic F) This commitment to political engagement at the local level also translated into the perception of their role as an academic: I’d always had this commitment to welfare and a fundamental belief that it’s the job of the intellectual to communicate the sort of public purse bit to that; you know, we’re paid from public money and they have to give something back, but it, you know, it’s not instrumental, it’s just [a] fundamental belief system. So that’s – that’s it; if social, if my life became just doing science I wouldn’t – I wouldn’t really want to do it. (Academic B) A key part of making a difference as a professional was the engagement not just with other scholars but also to provide academic assistance to pressure groups through research and membership of boards. Conclusion The aim of these interviews with social policy professors was to get a deeper understanding of the impact of research on policymaking over time, and to allow for critical reflection that included successes and failures as well as references to the work of colleagues. These features will necessarily not be found in the REF ICS narratives, which are, by definition, examples of success, and written in a linear structure, about impact within a set window, and with matching evidence. In the interviews

98

Dimension 4: Taking the long view: looking back over 40 years of Social Policy

respondents picked up on the issue of timing, namely that ideas often had been around for some time and had suddenly been integrated into policies at the point when they fitted the political priorities and narrative. Similarly, this could mean that any impact might not last, as policies can be amended or abolished by governments in the future. Also, impact could be in name only, where not much more than the name was transferred into policy, despite references to the academic work. It is also clear that the respondents see themselves as part of an epistemic community and frequently referred to each other’s work and influence. This community also had clear mentoring elements, with academics from the first generation playing a proactive role in introducing younger academics to policymaking through the establishment and collaboration with voluntary sector organisations, who then became lobbyists and knowledge brokers in their own right. The interviews facilitate a critical perspective onto to definition and operationalization of impact in the 2014REF. The focus on individuals or individual units within universities does not reflect the academic reality of epistemic communities and the resulting interconnectedness of the work of academics. The narrow construction of the value of an academic runs the danger of reducing academics to purveyors of their own knowledge only rather than a broader expertise in the area, which in particular, includes the ability to critically analyse and synthesise knowledge on a topic from a wide range of policy actors, and to thus contextualise their own work. The interviews also support the findings by Bastow et  al (2104) about the connectedness of Social Policy scholars with a range of voluntary sector bodies. However, it also clearly demonstrates that these academics have operated in a policy network context. Furthermore, the long view from the interviews has drawn attention to the fact that impact can be in name only, not be for a long duration or cause more harm than good. By the same token, the interviews also highlight what has moved out

99

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

of sight of policy-makers and practitioners, namely the often hugely important role of mentors and of core funding attached to a unit or centre allowing researchers to follow seemingly less important topics or new ideas without the need for a new application, ethical approval, etc. Looking ahead, some interviewees found it hard to be optimistic: ‘it’s not a successful time for Social Policy and it’s not a successful time for Social Policy to claim it’s influential’ (Academic A).

100

SEVEN Summary and conclusions

The introduction of impact into the 2014 Research Excellence Framework as an additional measure of academic work and worth was an attempt to judge universities on a more ‘balanced scorecard’ (Bence and Oppenheim, 2005: 151–152). The deliberately broad definition of impact for the REF2014 and demonstration via a qualitative case study with self-defined criteria of success both helped its applicability across disciplines and allowed researchers to include different types of impact in their ICS. And yet, the successful narratives found in many ICS stand in contrast to the more pessimistic assessment of the EBPM literature as well as the long view provided by the interviews on the contribution that academic research makes to policy. The contribution of this book is threefold: unlike most other studies of the REF2014 ICS, this book includes case studies from three different sub-panels, which in themselves capture several disciplines, and therefore allows for a comparison of how impact and academic identify are defined and presented. Therefore, it allows for an in-depth comparison across three key social sciences disciplines (see Chapter four). Second, the ICS are placed in an analytical framework that identifies different types of impact and impact pathways (see Chapter five) and places

101

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

them in the context of academic identities and different policy models (Chapter two). Finally, it provides a comparison across time based on interviews with Social Policy professors who are looking back over 40 years as well as analysing the relationship between research and policymaking (Chapter six). This long view allows the achievements as well as the serendipitous and superficial nature of impact to be placed into the context of different governments and stages of welfare state development. The main findings emerging from the comparison of the ICS The sample in this book contained between 30 and 40 case studies from the Sociology, Politics and International Relations and Social Policy and Social Work sub-panels. The selection of the case studies was based on the overall ranking of the ICS for the submitting units as a way of obtaining a selection of what is seen as excellent impact. When looking across three of the main social science disciplines a number of broader common trends become evident: The first, and rather striking, point is the duration, intensity and variety of the engagement of academics with policy actors across many of the case studies. Often the engagement with policy actors will have been ongoing for a decade or more and take the shape of different roles, from presenting findings to a range of policy audiences (those often become more high level and/or international as the research/er become more known in the area), to being part of advisory bodies and to carrying out small commissioned pieces of research and collaborations outside academia. The second trend is perhaps more surprising but no less welcome, namely, that there still seems to be a role for the academic as the expert on a topic over their role as producer of specific pieces or, even, programmes of research, which then form the basis of an ICS. The examples vary from conversations between academics and policymakers in Chatham House itself,

102

Summary and conclusions

or similar settings, to their many roles on advisory boards nationally and internationally. These advisory boards are often inter-disciplinary even if the ICS have not tended to be. The value and contribution of academics to the policymaking process is based on their expertise that is simultaneously broad and indepth and includes being able to quickly assimilate and evaluate new ideas and material. The third point is that the subject of the underlying research has been very varied as has its contribution. This ranged from accompanying or exploiting the potential of technological changes to normative questions such as support after family separation and to tackling complex problems such climate change. Therefore, the supposed barriers to impact that are said to exist for research in areas where ideological disputes continue, or where the likely solutions require inter-disciplinary work, do not seem to have mattered overly in this instance. The fourth trend picks up on a criticism of previous research exercises: that they tend to favour established universities and academics, with Russell Group universities featuring prominently in all three sub-panels in the REF2014 very much at the expense of post-1992 universities. The fifth trend is the dominance of not only the lone scholar but a marked absence of reference to collaborations (whether national, international or interdisciplinary) in the case studies. This deliberate positioning of the star academic and their host university was encouraged by the way impact was set up in the REF2014, which arguably undermines the value of the profession as a whole and negates existing epistemic communities. However, the extent to which the lone scholar roams differs by sub-panel, that is, more so in Politics and International Relations than in Sociology and least of all in Social Policy and Social Work. The latter is better characterised by submissions from research teams and research centres. A dimension not picked up by Dunlop (2018) is gender and seniority; and as with the lone scholar, the ICS in Politics

103

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

and International Relations seem to be much more likely to be submitted by a male professor than in the other two subpanels. While the difference is notable, it would require further investigation such as a comparison of the male/female ratio of students for the three main disciplines to warrant further comment. The seventh point to make is that many of the ICS claim incremental change and that radical change, in terms of direction or scale, is rare. The absence is due in part to the prescribed nature of the impact window, the single unit submissions and that UK-based academics are a part of, rather than the sole contributors to, a broader, international community. Furthermore, only submissions to the Social Policy and Social Work panel included examples of policy innovations, that is, wholesale policies being introduced. Finally, it is important to distinguish between impact agents and impact beneficiaries when assessing ICS in terms of their contribution to society. On the one hand impact agents are those able to make changes as policymakers or practitioners. Identifying them, their language and other priorities is therefore important in order to shape communication and for overall engagement to be constructive and ‘successful’. Impact beneficiaries on the other hand are those whose lives are improved as a result of the changes. For some of the ICS studied in this book, impact beneficiaries are often large groups of the population such as children in poverty, the elderly in need of care, prospective pensioners, or even all voters. Therefore, it is not an overstatement to say that the contribution of academics as captured by the impact submissions analysed here has improved the lives of many people in the UK and around the world. The relatively smooth running of the REF2014 and the deliberately high scoring of the ICS (Derrick, 2018) have all contributed to a far greater acceptance of impact since its announcement. Moreover, the exercise as a whole has led to the longstanding engagement activities and, thus, influence of a

104

Summary and conclusions

large number of academics becoming more visible to universities themselves (including colleagues) but also to government as a whole. Despite this, there are still critical voices that perhaps are more in line with the literature on EBPM. Boswell and Smith argue that the operationalisation of impact in the REF does not seem to be based on a clear understanding of the relationship between research and policy and thus risks ‘rewarding researchers for achieving impact’ through ‘adopting an arbitrary incentive system that is at best decoupled from research quality and, at worst, threatens the integrity and independence of social science’ (2018: 8). The assessment of impact as part of future REFs may well follow the same path as other elements, namely to become more robust but also more time-consuming and expensive and to favour established academics and institutions (see also Bastow et al, 2014). However, while the effect on academic behaviour probably will be strongest after the initial introduction in the 2014REF, policy-makers will also have noticed many academics being more keen to get involved and are likely to make use of this and thus embed academics more firmly into policy-making networks. It may therefore seem surprising that the interviews with the Social Policy professors reflected a rather more sceptical view of their own impact, or that of their discipline as a whole, and thus similar to the EBPM literature in their view of the ability of academics to influence policy-making. The reasons for this scepticism include some of the themes raised in the discussion of impact in the context of the REF2014, such as the frequently serendipitous and superficial uptake of research and the large gaps between the research and the impact that in some cases could amount to decades. In addition, the interviews highlight the difficulty of having positive impact during times of a government that seems hostile towards the welfare state (i.e. the academics speaking of having spent large periods of their careers fighting rear-guard action trying to prevent the dismantling of areas of the welfare state but also talk about radical rather than the more

105

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

incremental impact claimed in the ICS. However, one of the strongest themes coming across from the interviews is the sense of community. This takes various forms, such as more senior academics having acted as mentors and collaborators, not just on academic matters but also on how to achieve impact. This sense of community and joint endeavour also shines through when respondents try and pinpoint contributions towards policy changes. The serendipitous nature of timing for the take-up of research featured strongly in the interviews. Over the course of their academic careers, respondents were able to engage (or not) with governments of different persuasions in terms of their ideological stances on the welfare state. The careers of the academics interviewed for this book span an expansionist Labour government at the outset of their careers, a Conservative government concerned with the ballooning costs of the welfare state and very much influenced by neoliberal ideologies around the power of the free market and the importance of enforcing self-reliance of individuals, to a then much more pragmatic New Labour government that wanted to find out what works and were inclined to expand sections of the welfare state, though in a more conditional fashion than before. In other words, the economic circumstances as well as ideological orientation of governments affect the kind of influence academics can have on policy-making during that time. Furthermore, for this generation, the story of their impact is linked to the story of the end of the expansion, contraction and conditional expansion of the welfare state. For the REF2014, the period in question was mainly that of the later years of the New Labour government and then the beginning years of the Coalition government, which continued the implementation and general direction of a number of flagship New Labour policies such as the implementation of the Employment Support Allowance and Universal Credit, while also embarking on a programme of stringent cuts. The academics in my interview

106

Summary and conclusions

sample have taken different and, arguably, multiple routes in response to the change of government. A number of them focused on damage limitation and fought with voluntary section groups, often those they had helped set up, against the cuts and overall discourse around the welfare state. However, others, while also doing that, sought pastures new and green and instead focused on working with the EU, international organisations or the governments of other countries. In the context of Social Policy in the UK, an important part of timing in terms of a window of opportunity for impact has been the overall attitude of a government to the welfare state, which has implications for what kind and extent of impact can be achieved in different REF periods. However, when looking across the case studies from Sociology, Social Policy and Social Work and Politics and International Relations, the importance of the EU as a policymaking body is more prominent. Similarly, there seems to be greater collaboration with international organisations such as the UN, the World Bank and the ILO. And certainly, with organisations such as the ILO and the World Bank, the ideological leaning of individual governments matters less. With the EU and the UN that is slightly less so, but there is still a much greater sense of continuity overall in terms of staggered elections and focus on agreed topic areas than there would be at the national level in a country where the political system allows for radical policy swings between governments. Therefore, the argument for exploiting windows of opportunity in terms of ideological leanings applies less to international organisations than it does on the national level in the UK. Coming back, then, to the original question from my colleagues of ‘what is impact?’, I hope the analysis of the ICS has demonstrated that social sciences can affect the world outside the academy in many ways based on a wide variety of research projects and impact pathways. However, as the interviews suggest, this impact may be in name only, or may not last, and claims of impact should therefore be taken with a pinch of salt.

107

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Following the Stern review (2016) impact will form part of the next research assessment exercise, the REF 2021 (HEFCE, 2017a). The definition of impact has been aligned with that of the UK’s research councils and also pays greater attention to public engagement and impact on teaching (HEFCE, 2017a). However, the ratio of case studies to research submissions, as well as the timescales for when the research can have been carried out and when the impact can have taken place, are likely to be similar to the REF2014 (HEFCE, 2017a). In an ideal scenario, this exercise will further contribute to the visibility of academic contributions to the economy, society and culture in the UK and across the globe. Yet, the shortcomings of capturing the complexities of the influence of academics on policy-making due to the way impact is conceptualised and captured in the 2014REF (and 2021REF) will remain.

108

Appendix

A1: Descriptions of sub-panels for the REF2014 Sub-panel 21: Politics and International Relations

The UOA (Unit of Analysis) includes (but is not restricted to) comparative, area, national and subnational politics; public administration and policy studies; political behaviour and political sociology; 60 REF 01.2012 political theory and philosophy, including history of political thought; international relations, including strategic, war and peace studies, international history, international political economy and foreign policy analysis; methods in political studies; and higher education pedagogic research in politics and international studies. 15. Boundaries: The sub-panel expects submissions in this UOA from all areas of the discipline in the UOA descriptor, but recognises that some of the outputs submitted will cross disciplines; the sub-panel is confident of its ability to assess a wide range of interdisciplinary outputs. Descriptor: The UOA includes empirical and theoretical study of the social structures, cultures and everyday practices of societies, including styles and material standards of living, opinions, values and institutions. It covers all areas of social theory, historical and comparative studies,

109

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

and social research methodology (including qualitative and quantitative methods and visual methodologies), philosophy of social science, and research on pedagogy in sociology. The sub-panel also expects to consider sociological research in such interdisciplinary fields as criminology and sociolegal studies, media and cultural studies, demography, socio-linguistics, social psychology, psychosocial studies, social studies of science and technology (including science and technology policy), and lesbian, gay, bisexual, transgender and intersex studies.

Sub-panel 22: Social Policy and Social Work

The UOA [Unit of Analysis] covers all forms of research in social work, social policy and administration and criminology, including those in governmental, voluntary and community, private for profit and not for profit areas. Research includes: a.  Theory, methodology, empirical research, ethics and values, and pedagogy as they apply to social work, social care, social policy, criminology and criminal justice policy, gerontology and substantive issues in these areas of study. b. Comparative research and research into international institutions, policy and practice. c. Research that uses a range of disciplinary approaches including (but not exclusively) the following: business and management, demography, development studies, economics, education, geography, health studies, history, law, politics, psychology and sociology. d. Relevant links with other stakeholders, professionals, service users and carers. e.  Policymaking processes, practice, governance and management, service design, delivery and use, and inter-professional relationships. Boundaries: Social Work, Social Policy and Administration, and Criminology are essentially multidisciplinary subjects

110

Appendix

and are closely related to a range of other disciplines within the social sciences and more broadly. Appropriate methods will be used in cases of substantial overlap with other subpanels, as set out in Part 1, paragraphs 92–100. For the avoidance of doubt, it is recognised that criminological research may fall within the boundaries of Sub-panels 20 (Law), 22 (Social Work and Social Policy) and 23 (Sociology). All three sub-panels welcome such work, which will be assessed in accordance with the arrangements noted earlier, in particular making use of joint assessors and crossreferral as deemed appropriate by the sub-panels.

Sub-panel 23: Sociology

The sub-panel expects submissions in this UOA (Unit of Analysis) from all fields of sociological enquiry including, but not restricted to, research on cultures, economies, and polities; class, ethnicity, gender, sexuality, disability, and age, and their intersection; religion, education, health and medicine, family, media, welfare institutions, and work and employment; environment, technology, and climate change; the body, interpersonal and inter-group relations, violence; urban and rural issues; language and social interaction; political sociology, public policy, and social movements; political economy, globalisation, development, migration, and diaspora; comparative studies of societies of all kinds, including work on transnational structures and agencies, the EU, world systems. The subpanel welcomes work in social theory and the history of social thought. 20. As in previous research assessment exercises, work in interdisciplinary women’s studies may be submitted in this UOA, or may be cross-referred by other sub-panels to Sub-panel 23. Assessors will be appointed to consider the interdisciplinary aspects of women’s and

111

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

gender studies that fall outside the expertise of the subpanels. 21.  Work submitted in this UOA may overlap significantly with the remit of UOA 22 (Social Work and Social Policy). This arises from the large number of academic units that combine the constituent subject areas and that may make a combined submission in UOA 22 or UOA 23. It is anticipated that the use of joint assessors and cross-referral of parts of submissions may be required in order to ensure an appropriate assessment, in accordance with the arrangements in Part 1, paragraphs 92–100. For the avoidance of doubt, it is recognised that REF 01.2012 61 Part 2C Main Panel C criteria criminological research may fall within the boundaries of Sub-panels 20 (Law), 22 (Social Work and Social Policy) and 23 (Sociology). All three sub-panels welcome such work, which will be assessed in accordance with the arrangements noted earlier, in particular making use of joint assessors and crossreferral as deemed appropriate by the sub-panels. Source: HEFCE (2014a) Part  2C Main Panel  C criteria, pp 60–62 www.ref.ac.uk/2014/media/ref/content/pub/panel criteriaandworkingmethods/01_12_2C.pdf

112

Table A1: Ranking for impact case studies relative to overall ranking Social Policy and Social Work University

Overall REF ranking

Sociology University

Politics and International Relations Overall REF ranking

University

Overall REF ranking

2

York

1

Oxford

4

3

Cardiff

3

Essex

1

Oxford

1

Lancaster

4

Sheffield

3

UCL

11

Manchester

2

Keele

=33

Kent

5

Bristol

9

Leeds

=23

Loughborough

19

Roehampton

25

Kings

=15

=16

Bath

=6

Open University

LSE

2

Dundee

37

Oxford

5

Bradford

35

UEA

4

*

* *

York

8

Glasgow Caledonian

=35

*

* *

Strathclyde

3

South Wales

=26

*

*

*

*

Source: Times Higher Education, 2014

Appendix

113

LSE York

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

A2: Panel membership Main Panel C Chair Professor Dame Janet Finch

University of Manchester

Members Professor Cara Aitchison

University of St Mark and St John

Professor Trevor Barnes

University of British Columbia

Professor Frans Berkhout

King’s College London

Professor Hastings Donnan

Queen’s University Belfast

Professor Gillian Douglas

Cardiff University

Professor Colin Hay

University of Sheffield

Professor Herbert M. Kritzer

University of Minnesota

Professor Peter Neary

University of Oxford

Professor Wim van Oorschot

Leuven University

Professor Jone Pearce

University of California, Irvine

Changed Inst May 2013

Changed Inst Aug 2013

Until Mar 2014

Professor Alan Penn

University College London

Professor Michael Pidd

Lancaster University

Professor Andrew Pollard

UCL Institute of Education

Professor Keith Richards

University of Cambridge

Mr Mark Hunter Robson

Bank of England

Ms Sue Rossiter

Charities Evaluation Services

Changed Org Nov 2013

Professor John Scott

(Formerly) University of Plymouth

Retired from Inst Apr 2014

Professor Peter Taylor-Gooby University of Kent Dr Martin Walsh

Oxfam GB

114

Appendix

Professor Paul Wiles

Board member of Food Standards Agency and NatCen

Changed Org Jan 2014

Ms Teresa Williams

Nuffield Foundation

Joined Feb 2013

Ms Sharon Witherspoon

Nuffield Foundation

Until Dec 2012

Observer Dr Fiona Armstrong

Economic and Social Research Council

Mr Iain Jones

Economic and Social Research Council

Panel Advisers – Assessment Phase 2014 Mrs Michelle Double

University of Leeds

Mrs Deborah McClean

University of Sheffield

Mrs Katherine McKen

University of Bath

Dr Michael Wykes

University of Exeter

Sub-panel 21: Politics and International Studies Chair Professor Colin Hay

University of Sheffield & Sciences Po, Paris

Deputy chair Professor Vivien Lowndes

University of Nottingham

Members Professor Richard Bellamy

University College London, EUI Florence, Italy

Professor Chris Brown

London School of Economics and Political Science

Professor Roland Dannreuther University of Westminster Professor John Dumbrell

University of Durham

115

Until Aug 2011

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Professor James Dunkerley

Queen Mary University of London

Professor Richard English Professor Andrew Hurrell Professor Vivienne Jabri Professor Charlie Jeffery Mr Peter Kellner Professor Colin McInnes Professor Iain McLean Professor Nicola Phillips Professor Shirin Rai Rt Hon Peter Riddell Professor David Sanders Professor Judith Squires Professor Jonathan Tonge Mr Martin Williamson

University of St Andrews University of Oxford King’s College London University of Edinburgh YouGov Plc Until Oct 2011 Aberystwyth University University of Oxford University of Sheffield University of Warwick Institute for Government Until Apr 2012 University of Essex University of Bristol University of Liverpool University of Exeter Changed Inst Nov 2013

Assessors Mr Stephen Blakeley

KPMG

Joined Jul 2013

Dr Pilar Domingo

Overseas Development Institute

Joined Dec 2013

Mr Paul Evans

House of Commons

Joined Jul 2013

Dr Catherine Fieschi

Counterpoint

Joined Nov 2013

Professor Peter John

University College London

Joined Jun 2013

Mr Guy Lodge

Institute for Public Policy Research

Joined Jun 2013

Ms Juliet Lyon

Prison Reform Trust

Joined Oct 2013

Professor Trevor Taylor

Royal United Services Institute

Joined Jun 2013

Secretariat – Assessment Phase 2014 Dr Michael Wykes

University of Exeter

Ms Joanne Lakey

UCL Institute of Education

116

Appendix

Sub-panel 22: Social Work and Social Policy Chair Professor Peter Taylor-Gooby

University of Kent

Deputy chairs Professor Loraine Gelsthorpe

University of Cambridge

Professor Imogen Taylor

University of Sussex

Members Professor Pete Alcock

University of Birmingham

Professor Judith Allsop

University of Lincoln

Professor Saul Becker

University of Birmingham

Changed Inst Aug 2014 Until Apr 2014

Dr Robert Berkeley

Runnymede Trust

Professor Hugh Bochel

University of Lincoln

Professor Alison Bowes

University of Stirling

Professor Michele Burman

University of Glasgow

Professor John Carpenter

University of Bristol

Professor Nick Ellison

University of York

Ms Alison Garnham

Child Poverty Action Group

Professor Barry Goldson

University of Liverpool

Professor Hilary Graham

University of York

Professor John Hills

London School of Economics and Political Science

Mr David Johnson

Department for Energy and Climate Change

Professor Geraldine Macdonald Queen’s University Belfast Professor Jane Millar

University of Bath

117

Changed Inst Sep 2013

Changed Org Jul 2013

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Professor Julia O’Connor

University of Ulster

Professor Judith Phillips

Swansea University

Professor Roger Smith

University of Durham

Professor Peter Squires

University of Brighton

Professor Nicky Stanley

University of Central Lancashire

Professor Susan White

University of Birmingham

Dr Karl Wilding

National Council for Voluntary Organisations

Assessors Professor Marian Barnes

University of Brighton

Joined Jun 2013

Professor Tim Blackman

The Open University

Joined Jun 2013

Professor Ben Bowling

King’s College London

Joined Jan 2013 Joint with SP20 and SP23

Dr Alex Burfitt

National Audit Office

Joined Jul 2013

Professor Roger Burrows

Goldsmiths, University of London

Joined Jun 2013

Professor Jane Falkingham

University of Southampton

Joined Jun 2013

Dr Peter Gilleece

Northern Ireland Policing Board

Joined Nov 2013

Professor Ravi Kohli

University of Bedfordshire

Joined Jun 2013

Professor Robert MacDonald

Teesside University

Joined Jun 2013

Ms Kate Stanley

National Society for the Joined Jun 2013 Prevention of Cruelty to Children

Mr Nick Timmins

Institute for Government and The King’s Fund

Joined Sep 2013

Mr Bernard Walker

Director of Social Services, Wigan

Joined Jun 2013

Until Mar 2014

118

Appendix

Secretariat – Assessment Phase 2014 Mrs Michelle Double

University of Leeds

Miss Jennifer Hulin

Cardiff University

Sub-panel 23: Sociology Chair Professor John Scott

(Formerly) University of Plymouth

Retired from Inst Apr 2014

University of Oxford

Changed Inst Nov 2012

Deputy chair Professor Mary Daly Members Professor Sara Arber

University of Surrey

Professor Chetan Bhatt

London School of Economics and Political Science

Professor Eamonn Carrabine University of Essex Professor Nickie Charles

University of Warwick

Professor Graham Crow

University of Edinburgh

Dr Sara Delamont

Cardiff University

Professor Tia DeNora

University of Exeter

Mrs Barbara Doig

Barbara Doig AcSS Independent Consultant

Professor Linda McKie

University of Durham

Professor Mike Savage

London School of Economics and Political Science

Dr William Solesbury

William Solesbury Associates

Professor John Solomos

University of Warwick

Professor Liz Stanley

University of Edinburgh

119

Changed Inst Mar 2013

Changed Inst Jan 2013

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Professor John Thompson

University of Cambridge

Professor Sylvia Walby

Lancaster University

Professor Sandra Walklate

University of Liverpool

Professor Alan Warde

University of Manchester

Professor Nira Yuval-Davis

University of East London

Assessors Professor Ben Bowling

King’s College London

Joined Jan 2013 Joint with SP20 and SP22

Ms Helen Chambers

Inspiring Scotland

Joined Jun 2013

Dr Omar Khan

Runnymede Trust

Joined Jun 2013

Dr Ceridwen Roberts

University of Oxford

Joined Jun 2013

Professor Nikolas Rose

King‘s College London

Joined Jun 2013

Mr David Walker

Guardian Public

Joined Jun 2013

Secretariat – Assessment Phase 2014 Mrs Deborah McClean

University of Sheffield

Miss Gillian Weale

University of Cambridge

Source: HEFCE, 2015a

120

References

Alcock, P (2003) ‘The subject of Social Policy’, in P Alcock, A Erskine and M May (eds) The Students’ Companion to Social Policy. Oxford: Social Policy Association, Blackwell. Alcock, P, Glennester, H, Oakley, A and Sinfield, A (2001) Welfare and Well-being: Richard Titmuss’ Contribution to Social Policy. Bristol: Policy Press. Bastow, S, Dunleavy, P and Tinkler, J (2014) The Impact of the Social Sciences: How Academics and their Research Make a Difference. London: Sage. Bence, V and Oppenheim, C (2005) ‘The evolution of the UK’s Research Assessment Exercise: Publications, performance and perceptions’, Journal of Educational Administration and History, 37(2), 137–155. Biri, D, Oliver, KA and Cooper, A (2014) What is the Impact of BEAMS Research? An Evaluation of REF Impact Case Studies from UCL BEAMS. London: University College London. Blume, S.S (1977) ‘Policy as theory: A framework for understanding the contribution of social science to welfare policy 1’ Acta Sociologica, 20(3), 247–262. Bochel, H and Duncan, S (2007) Making Policy in Theory and Practice. Bristol: Policy Press. Bornmann, L (2013) ‘What is societal impact of research and how can it be assessed? A literature review’, Journal of the American Society for Information Science and Technology, 64(2), 217–233.

121

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Bornmann, L and Marx, W (2014) ‘How should the societal impact of research be generated and measured? A proposal for a simple and practicable approach to allow interdisciplinary comparisons’, Scientometrics, 98(1), 211–219. Brewer, JD (2011) ‘The impact of impact’, Research Evaluation, 20(3), 255–256. British Academy (2008) Punching our Weight: The Humanities and Social Sciences in Public Policy Making. London: British Academy. Brouwer, DC and Squires, CR (2003) ‘Public intellectuals, public life, and the university’, Argumentation and Advocacy, 39(3), 201–213. Brown, C (2012) ‘Exploring the concepts of knowledge adoption and conceptual impact: Impact for educational research submissions to the Research Excellence Framework 2014’, Education, Knowledge and Economy, 5(3), 137–154. Bulmer, M, Lewis, J and Piachaud, D (1989) The Goals of Social Policy. London: Allen and Unwin. Bulmer, M (2015) The Uses of Social Research (Routledge Revivals): Social Investigation in Public Policy-Making. London: Routledge. Burrows, R (2012) ‘Living with the h‑index? Metric assemblages in the contemporary academy’, The Sociological Review, 60(2), 355–372. Cairney, P (2016) The Politics of Evidence-Based Policy Making. London: Palgrave Macmillan. Cairney, P and Kwiatkowski, R (2017) ‘How to communicate effectively with policymakers: Combine insights from psychology and policy studies’, Palgrave Communications, 3(1), 37. Cairney, P and Oliver, K (2018) ‘How should academics engage in policymaking to achieve impact?’, Political Studies Review. www. doi.org/10.1177/1478929918807714 Caplan, N (1979) ‘The two-communities theory and knowledge utilization’, American Behavioral Scientist, 22(3), 459–470. Collini, S (2012) What are Universities For?, London: Penguin. Collini, S (2013) ‘Sold out’, London Review of Books, 35(20). www. lrb.co.uk/v35/n20/stefan-collini/sold-out Dahrendorf, R (1995) LSE: A History of the London School of Economics and Political Science 1895–1995. Oxford: Oxford University Press.

122

References

Deacon, A (1993) ‘Richard Titmuss: 20 years on’, Journal of Social Policy, 22(2), 235–242. Denicolo, P (ed) (2013) Achieving Impact in Research. London: Sage. Derrick, G (2018) The Evaluators’ Eye: Impact Assessment and Academic Peer Review. London: Palgrave Macmillan. Donnison, D (1979) ‘Social policy since Titmuss’, Journal of Social Policy, 8(2), 145–156. Donnison, D (2016) ‘On tap and not on top. Academic Contributions to the Analysis of Social Problems and the Formulation of Social Policy’, talk at the University of Glasgow, Tuesday 9  February. https://www.gla.ac.uk/schools/socialpolitical/research/politics/ stevensontrust/newsandevents/headline_442787_en.html Donovan, C (2011) ‘Impact is a strong weapon for making an evidencebased case for enhanced research support but a state-of-the-art approach to measurement is needed’, LSE Impact Blog, 22 August. https://blogs.lse.ac.uk/impactofsocialsciences/2011/08/22/ impact-strong-weaponevidence-based-case-for-enhanced-researchsupport-but-a-state-of-the-art-approach-to-measurement-isneeded Dover, R and Goodman, MS (2018) ‘Impactful scholarship in intelligence: a public policy challenge’, British Politics, 13(3), 374–391. Duncan, S (2006) ‘Towards evidence inspired policy’, Social Sciences, 61, 10–11. Duncan, S and Harrop, A (2006) ‘A user perspective on research quality’, International Journal of Research Methodology, 9(2), 159–174. Dunlop, CA (2018) ‘The political economy of politics and international studies impact: REF2014 case analysis’, British Politics, 13(3), 270–294. Eyal, G and Buchholz, L (2010) ‘From the sociology of intellectuals to the sociology of interventions’, Annual Review of Sociology, 36, 117–137. Fordham, FT and Noble, B (2016) ‘What is in a name? Evidence of impact in palliative and end-of-life care in the 2014 REF is difficult to find’, BMJ Supportive & Palliative Care, 6, 248–250.

123

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Fraser, D (1992) The Evolution of the British Welfare State: A History of Social Policy Since the Industrial Revolution. London: Macmillan International Higher Education. French, RD (2019) ‘Is it time to give up on evidence-based policy? Four answers’, Policy & Politics, 47(1), 151–168. Gilroy, P and McNamara, O (2009) ‘A critical history of research assessment in the United Kingdom and its post-1992 impact on education’, Journal of Education for Teaching, 35(4), 321–335. Glendinning, C and Millar, J (1992) Women and Poverty in Britain: the 1990s. London: Harvester Wheatsheaf. Glennerster, H (2014) ‘Richard Titmuss: Forty years on’, Case Paper 180, London: LSE. http://sticerd.lse.ac.uk/dps/case/cp/ casepaper180.pdf Glennester, H (2009) ‘Social Policy: Now and then – a response’, Social Policy & Administration, 43(7), 750–753. Grant, J (2015) ‘The nature, scale and beneficiaries of research impact: An initial analysis of Research Excellence Framework (REF) 2014 impact case studies’, Research Report 2015/01, King’s College, London, UK. https://www.kcl.ac.uk/policy-institute/assets/ref-impact.pdf Griffiths, D (2010) ‘Academic influence amongst the UK public elite’, Sociology, 44(4), 734–750. Haas, P.M (1992) ‘Introduction: epistemic communities and international policy coordination’, International organization, 46(1), 1–35. Halsey, AH (2004) A History of Sociology in Britain: Science, Literature, and Society. Oxford: Oxford University Press. Haux, T (2013) ‘Understanding employment barriers for lone parents in Great Britain: Research gaps and missed opportunities’, Social Policy & Administration, 47(4), 468–482. Haux, T (2018) ‘What is “impact”? Learning from examples across the professional life course’, in C Needham, E Heins and J Rees (eds) Social Policy Review 30: Analysis and Debate in Social Policy, 2018, p 181. Hayton, R (2018) ‘The impact agenda and the study of British politics’, British Politics, 13(3), 361–373.

124

References

Head, BW (2008) ‘Three lenses of evidence-based policy’, Australian Journal of Public Administration, 67(1), 1–11. Head, BW (2010) ‘Reconsidering evidence-based policy: Key issues and challenges’, Policy and Society, 29(2), 77–94. Head, BW (2016) ‘Toward more “evidence-informed” policy making?’, Public Administration Review, 76(3), 472–484. HEFCE (Higher Education Funding Council for England) (2008) About us. http://www.rae.ac.uk/aboutus/history.asp HEFCE (Higher Education Funding Council for England) (2010) Research Excellence Framework Impact Pilot Exercise: Findings of the Expert Panels. A report to the UK higher education funding bodies by the chairs of the impact pilot panels. www.ref.ac.uk/media/ref/content/ pub/researchexcellenceframeworkimpactpilotexercisefindingsofthe expertpanels/re01_10.pdf HEFCE (Higher Education Funding Council for England) (2011, updated in 2012) Assessment Framework and Guidance on Submissions, London. www.ref.ac.uk/media/ref/content/pub/assessment frameworkandguidanceonsubmissions/GOS%20including%20 addendum.pdf HEFCE (Higher Education Funding Council for England) (2012) Assessment Framework and Guidance on Submissions, London. www. ref.ac.uk/2014/media/ref/content/pub/assessmentframework andguidanceonsubmissions/GOS%20including%20addendum.pdf HEFCE (Higher Education Funding Council for England) (2014a) Assessment Criteria and Level Definitions. www.ref.ac.uk/panels/ assessmentcriteriaandleveldefinitions/impactpilotpanels HEFCE (Higher Education Funding Council for England) (2014b) Impact Pilot Exercise. www.ref.ac.uk/2014/background/pilot/ HEFCE (Higher Education Funding Council for England) (2015a) Panel Membership. www.ref.ac.uk/2014/panels/panelmembership/ HEFCE (Higher Education Funding Council for England) (2015b) 2014 REF Impact Case Studies, https://impact.ref.ac.uk/casestudies/ HEFCE (Higher Education Funding Council for England) (2017a) Initial Decisions on the Research Excellence Framework 2021, London. www.ref.ac.uk/media/ref,2021/downloads/REF2017_01.pdf

125

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

HEFCE (Higher Education Funding Council for England) (2017b) What is the REF?, London. www.ref.ac.uk/about/whatref/ Heinrich, CJ (2007) ‘Evidence-based policy and performance management: Challenges and prospects in two parallel movements’, The American Review of Public Administration, 37(3), 255–277. Hicks, D (2012) ‘Performance-based university research funding systems’, Research Policy, 41(2), 251–261. Holmwood, J (2011) ‘The impact of “impact” on UK social science’, Methodological Innovations Online, 6(1), 13–17. Holmwood, J and McKay, S (2015) ‘As REF 2014 goes by: A fight for cash and glory…(with apologies to Casablanca)’, The Sociological Review Blog, 26. www.parliament.uk/documents/post/ POST_Role%20of%20Research%20in%20UK%20Parliament%20 2017.pdf Jacobs, RN and Townsley, E (2011) The Space of Opinion: Media Intellectuals and the Public Sphere, Oxford: Oxford University Press. Jones, MM, Manville, C and Chataway, J (2017) ‘Learning from the UK’s research impact assessment exercise: A case study of a retrospective impact assessment exercise and questions for the future’, The Journal of Technology Transfer. https://doi.org/10.1007/ s10961-017-9608-6 Jonkers, K and Zacharewicz, T (2016) ‘Research performance based funding systems: A comparative assessment’, Institute for Prospective Technological Studies, Joint Research Centre. http://publications.jrc. ec.europa.eu/repository/bitstream/JRC101043/kj1a27837enn.pdf Kelly, D, Kent, B, McMahon, A, Taylor, J, and Traynor, M (2016) ‘Impact case studies submitted to REF 2014: The hidden impact of nursing research’, Journal of Research in Nursing, 21(4), 256–268. Kenny, C, Rose, C, Tyler, C and Blackstock, J (2017) The Role of Research in Parliament. London: HMSO. Khazragui, H and Hudson, J (2014) ‘Measuring the benefits of university research: Impact and the REF in the UK’, Research Evaluation, 24(1), 51–62.

126

References

King’s College London and Digital Science (2015) The Nature, Scale and Beneficiaries of Research Impact: An Initial Analysis of Research Excellence Framework (REF) 2014 Impact Case Studies. London: King’s College London and Digital Science. Klocker, N and Drozdzewski, D (2012) ‘Commentary: Career progress relative to opportunity: how many papers is a baby “worth”?’, Environment and Planning A, 44(6), 1271–1277. Kurzman, C and Owens, L (2002) ‘The sociology of intellectuals’, Annual Review of Sociology, 28(1), 63–90. Kwok, J (2013) ‘Altmetrics make their mark’, Nature, 500, 491–493. Lindblom, C.E (1959) ‘The science of muddling through’, Public Administration Review, 19(2), 79–88. Lindblom, CE and Cohen, DK (1979) Usable Knowledge: Social Science and Social Problem Solving. New Haven, CT: Yale University Press. Lupton, R and Burchardt, T (eds) (2016) Social Policy in a Cold Climate: Policies and their Consequences Since the Crisis. Bristol: Policy Press. McGettigan, A (2013) The Great University Gamble: Money, Markets and the Future of Higher Education. London: Pluto Press. McKay, S (2012) ‘Social policy excellence – peer review or metrics? Analyzing the 2008 research assessment exercise in social work and social policy and administration’, Social Policy & Administration, 46(5), 526–543. Manville, C, Guthrie, S, Henham, ML, Garrod, B, Sousa, S, Kirtley, A and Ling, T (2015) Assessing Impact Submissions for REF 2014: An Evaluation. Cambridge: RAND Europe. Marcella, R, Lockerbie, H and Bloice, L (2016) ‘Beyond REF 2014: The impact of impact assessment on the future of information research’, Journal of information Science, 42(3), 369–385. Martin, BR (2011) ‘The Research Excellence Framework and the “impact agenda”: Are we creating a Frankenstein monster?’, Research Evaluation, 20(3), 247–254. Mead, LM (2015) ‘Only connect: Why government often ignores research’, Policy Sciences, 48(2), 257–272.

127

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Morgan Jones, M, Castle-Clarke, S, Manville, C, Gunashekar, S and Grant, J (2013) ‘Assessing Research Impact. Research Report 278’. Cambridge: Rand. https://www.rand.org/pubs/research_reports/ RR278.html Morrish, L (2014) ‘Against REFonomics: Quantification cannot satisfy the demands of rationality, equity and tolerability’. http:// blogs.lse.ac.uk/impactofsocialsciences/2014/12/22/refonomicsand-reformations/ Morton, S (2015a) ‘Progressing research impact assessment: A “contributions” approach’, Research Evaluation, 24(4), 405–419. Morton, S (2015b) ‘Creating research impact: The roles of research users in interactive research mobilisation’, Evidence & Policy: A Journal of Research, Debate and Practice, 11(1), 35–55. Newman, J (2017) ‘Debating the politics of evidence-based policy’, Public Administration, 95(4), 1107–1112. Newman, J and Head, B (2015) ‘Beyond the two communities: A reply to Mead’s “why government often ignores research”’, Policy Sciences, 48(3), 383–393. Newman, J, Cherney, A and Head, BW (2016) ‘Do policy makers use academic research? Reexamining the “two communities” theory of research utilization’, Public Administration Review, 76(1), 24–32. Nixon, J (1996) ‘Professional identity and the restructuring of higher education’, Studies in Higher Education, 21(1), 5–16. Nixon, J, Marks, A, Rowland, S, and Walker, M (2010) ‘Towards a new academic professionalism: A manifesto of hope’, British Journal of Sociology of Education, 22(2), 227–244. Nutley, S, Walter, I, Davies, HTO (2007) Using Evidence: How Research can Inform Public Services. Bristol: Policy Press. Oakley, A (2014) Father and Daughter: Patriarchy, Gender and Social Science. Bristol: Policy Press. Osborne, T (2004) ‘On mediators: Intellectuals and the ideas trade in the knowledge society’, Economy and Society, 33(4), 430–447. Page, R (2010) ‘The Changing Face of Social Administration’, Social Policy & Administration, 44(3), 326–342.

128

References

Pahl, R (1977) ‘Playing the rationality game: the sociologist as a hired expert’ in C Bell and H Newby Doing Sociological Research, pp 130–148. Parkhurst, J (2017) The Politics of Evidence: From Evidence-based Policy to the Good Governance of Evidence. London: Taylor & Francis. Pawson, R. and Tilley, N (1997) Realistic Evaluation. London: Sage. Penfield, T, Baker, M, Scobie, R and Wykes, M (2014) ‘Assessment, Evaluations, and Definitions of Research Impact: A Review’, Research Evaluation, 23, 21–32. POST (Parliamentary Office for Science and Technology) (2018) Research and Impact on Legislatures. London: UK Parliamentary Office for Science and Technology. www.parliament.scot/ Research_Impact_in_Legislatures.pdf Powell, M (2010) ‘Social policy and administration: Journal and discipline’, Social Policy & Administration, 40(3), 233–249. Reed, MS (2016) The Research Impact Handbook. Huntly: Fast Track Impact. REF 2014 (no date) REF2014 Impact Case Studies. http://impact.ref. ac.uk/CaseStudies/FAQ.aspx Rein, M (1980) ‘Methodology for the study of the interplay between social science and social policy’, International Social Science Journal, 22(2), 361–368. Ringer, FK (1992) Fields of Knowledge: French Academic Culture in Comparative Perspective, 1890–1920. Toronto: University of Toronto Press. Rose, H (1981) ‘Re-reading Titmuss: the sexual division of welfare’, Journal of Social Policy, 10(4), 477–501. Ross, F, Morrow, E (2016) ‘Mining the REF impact case studies for lessons on leadership, governance and management in higher education’, Impact of Social Sciences Blog. https://blogs.lse.ac.uk/ impactofsocialsciences/2016/06/08/leadership-governance-andmanagement-research-mining-the-ref-impact-case-studies/ Ryan-Flood, R and Gill, R (eds) (2013) Secrecy and Silence in the Research Process: Feminist Reflections. London: Routledge.

129

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Sanderson, I (2002) ‘Evaluation, policy-learning and evidence-based policy-making’, Public Administration, 80(1), 1–22. Sayer, D (2015) ‘Why did REF2014 cost three times as much as the RAE? Hint: It’s not just because of the added impact element’, LSE Impact of the Social Sciences Blog. http://blogs.lse.ac.uk/ impactofsocialsciences/2015/08/03/why-did-the-2014-ref-costthree-times-as-much-as-the-2008-rae-hint-its-not-just-becauseof-impact/ Selby, J (2018) ‘Critical international relations and the impact agenda’, British Politics, 13(3), 332–347. Smith, K (2010) ‘Research, policy and funding: Academic treadmills and the squeeze on intellectual spaces’, British Journal of Sociology, 61(1), 176–195. Smith, K (2013) Beyond Evidence Based Policy in Public Health: The Interplay of Ideas. London: Springer. Smith, K and Haux, T (2017) ‘Evidence-based policy-making (EBPM)’, in B Greve (ed), Handbook of Social Policy Evaluation, London: Palgrave, pp 141–160. Smith, KE and Joyce, KE (2012) ‘Capturing complex realities: understanding efforts to achieve evidence-based policy and practice in public health’, Evidence & Policy: A Journal of Research, Debate and Practice, 8(1), 57–78. Smith, K and Stewart, E (2016) ‘The uneven impacts of research impact: Adjustments needed to address the imbalance of the current impact framework’, LSE Impact of the Social Sciences Blog. https:// blogs.lse.ac.uk/impactofsocialsciences/2016/07/25/the-unevenimpacts-of-research-impact/ Smith, KE and Stewart, E (2017) ‘We need to talk about impact: Why social policy academics need to engage with the UK’s research impact agenda’, Journal of Social Policy, 46(1), 109–127. Smith, S, Ward, V and House, A (2011) ‘“Impact” in the proposals for the UK’s Research Excellence Framework: Shifting the boundaries of academic autonomy’, Research Policy, 40(10), 1369–1379.

130

References

Solesbury, W (2001) ‘Evidence-based policy: Whence it came and where it’s going’, UK Centre for Evidence-based Policy and Practice Working Paper  1. www.kcl.ac.uk/sspp/departments/ politicaleconomy/research/cep/pubs/papers/assets/wp1.pdf Stern, N (2016) Building on Success and Learning from Experience: Independent Review of the Research Excellent Framework. London: HMSO. Stevens, A (2011) ‘Telling policy stories: An ethnographic study of the use of evidence in policy-making in the UK’, Journal of Social Policy, 40(2), 237–256. Stoker, G and Evans, M (eds) (2016) Evidence-based Policy Making in the Social Sciences: Methods that Matter. Bristol: Policy Press. Syed, J and Davies, J (2016) ‘Diversity in the authorship of journal articles and REF impact case studies: How are UK business schools shaping up?’, BAM (British Academy of Management) 2016 Conference Proceedings. Thane, P (1996) Foundations of the Welfare State (2nd edn). London: Routledge. THE (Times Higher Education) (2014) Research Excellence Framework 2014: Institutions Ranked by Subject. www.timeshighereducation. com/sites/default/files/Attachments/2014/12/17/x/o/z/sub14-01.pdf THE (Times Higher Education) (2017) ‘Jo Johnson plans for KEF to measure knowledge exchange’, Times Higher Education, 12 October. www.timeshighereducation.com/jo-johnson-plans-kef-measureknowledge-exchange Timmins, N (2001) The Five Giants: A Biography of the Welfare State. London: HarperCollins. Tinkler, J (2012) ‘The REF doesn’t capture what government wants from academics or how academic impact on policymaking takes place’, British Politics and Policy at LSE, blog entry. http://eprints. lse.ac.uk/43909/ Vostal, F (2015) ‘Academic life in the fast lane: The experience of time and speed in British academia’, Time & Society, 24(1), 71–95.

131

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

Wade, P (2013) ‘The REF’s narrow definition of impact ignores historical role of teaching in relation to the social impact of the   university’, Impact of the Social Sciences Blog, 12 August. http://blogs. lse.ac.uk/impactofsocialsciences/2013/04/12/ref-impact-the-stateand-the­universities-historical-perspectives/ Watermeyer, R and Hedgecoe, A (2016) ‘Selling “impact”: Peer reviewer projections of what is needed and what counts in REF impact case studies. A retrospective analysis’, Journal of Education Policy, 31(5), 651–665. Wehrens, R, Bekker, M and Bal, R (2011) ‘Coordination of research, policy and practice: a case study of collaboration in the field of public health’, Science and Public Policy, 38(10), 755–766. Weiss, C (1979) ‘The many meanings of research utilization’, Public Administration Review, 39(5), 426–431. Weiss, C (1982) ‘Policy research in the context of diffuse decision making’, The Journal of Higher Education, 53(6), 619–639. Welshman, J (2004) ‘The unknown Titmuss’, Journal of Social Policy, 33(2), 225–247. Wilding, P (2009) ‘Social Policy: Now and then’, Social Policy & Administration, 43(7), 736–749. Williams, F (1989) Social policy: a critical introduction: issues of race, gender, and class. Cambridge: Polity. Wilkinson, C (2017) ‘Evidencing impact: A case study of UK academic perspectives on evidencing research impact’, Studies in Higher Education, 44:1, 72–85. https://doi.org/10.1080/0307507 9.2017.1339028 Williams, K and Grant, J (2018) ‘A comparative review of how the policy and procedures to assess research impact evolved in Australia and the UK’, Research Evaluation, 27(2), 93–105. Yarrow, E and Davies, J (2018) ‘The gendered impact agenda – how might more female academics’ research be submitted as REF impact case studies?’, Impact of the Social Sciences Blog, 8  March. http://eprints.lse.ac.uk/89047/1/impactofsocialsciences-2018-0308-the-gendered-impact-agenda-how.pdf

132

References

Zacharewicz, T, Lepori, B, Reale, E and Jonkers, K (2018) ‘Performance-based research funding in EU Member States: A comparative assessment’, Science and Public Policy, 46(1), 105–115.

133

Index

anti-social behaviour 80 Atkin, Karl 79 atrocities, mass 67–8, 77–8 attitudes and habits and consumption 52, 53 towards research 16–17

Note: Page numbers for tables appear in italics. 1960s universities 49

A academic freedom 10 academic identity 97–8 academics collaboration between 55, 56–7, 88–9, 95–6, 106 and industry 1 and the media 58–9 and policy-makers 17–18, 39, 99, 102, 107 of social policy 31–3, 34, 83–100, 105–7 see also universities Adult Social Care Outcomes Toolkit (ASCOT) 81 advisory boards 103 advocates 82 Africa 39, 80–1 ageing, active 93 AIDS 80–1 Anti-Mafia Commission 68

B Bangladesh 65 basic income (BI) 67 Bastow et al 15, 25, 27, 43, 86, 94, 99 Bath, University of 45, 65 beneficiaries, impact 78–81, 82, 104 biobanking 51, 52 biomedical science 51–2 biosensors 79–80 Biri et al 78 bisexuality 39, 40 Boswell, C 104 BPIX 58 Bradford, University of 31 Bradshaw, Jonathan 39 Brewer, JD 10

135

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

between universities and industry 1 communication 17–18 community, broader epistemic 95–7, 99, 106 conceptual contributions 36, 40–2, 43–4, 59 confidentiality 51 Conflict Prevention Framework, ECOWAS 42 consumption 52, 53 ‘co-production’ of research 17–18 corruption 38 counterfactuals 23 crime, organised 68–9 Criminology 28, 75 Crucible Centre for Human Rights Research 69 ‘cultural gap’ 13 Curtice, John 56

Britain 77 see also UK British Election Study (BES) 57, 58 British Politics 26–7 see also Politics B-SEM (Bristol Social Exclusion Matrix) 41

C CAF (‘Research Contributions Framework’) 62–3 Cairney, P 12, 74 Caplan, N. 13, 17 ‘centrality’ of the research contribution 23 change 50, 53, 96, 104 child poverty 39, 85, 88–9, 91, 92, 96 child protection 57 children with AIDS 80–1 with anti-social behaviour 80 death of 57 and trauma recovery 64 and well-being 37, 39, 88–9 Chubb, J 23 climate change 52, 53 Cohen, DK 2 collaboration between academics 55, 56–7, 88–9, 95–6, 106 between academics and policymakers 17–18, 39, 99, 102, 107 between academics and the media 58–9

D data capture 4, 70–1, 88–9 data collection 37, 41 data stewardship organisations (DSOs) 51 death, child 57 democracy 37, 38 deprivation 37, 39, 41 Derrick, G 55, 58 disabilities, employees with 70–1 disability benefits for older people 66 ‘distributed agency’ 46 drug policy, illicit 67 Dundee, University of 64 Dunlop, CA 24–5, 35, 40–1, 46, 55, 103

136

INDEX

Firth, Prof David 56 Fisher, Dr Stephen 56 fit 15–16 forced marriage (FM) 70 freedom, academic 10 French, RD 12 funding regional 71 research 43–5, 50

E East Anglia, University of 55, 57, 65–6 EBPM (evidence-based policymaking) 2, 7–8, 11–19 ECOWAS (Economic Community of West African States) 42 election results 56 employees with disabilities 70–1 employment policies 71 engagement and exchange 18 with policy actors 102 political 97–8 public 58–9 environmental change 53 Equality and Human Rights Commission (EHRC) 71 Equality Performance Indicators (UK) 70–1 Essex, University of 38, 58 ethics, military 69 EU (European Union) 40, 41, 52, 53, 67–8, 71, 107 European Foreign Policy Unit (EFPU) 67–8 evidence-based policymaking (EBPM) 2, 7–8, 11–14, 16, 17–19 evidence-informed policymaking 19 expert, role for academic as 32, 52, 53, 56, 70, 102–3

G Gardner, Frances 80 Geddes, Professor Andrew 53 gender and human development 42 inequalities 71 and lead grant applicants 48, 103–4 genocide 67–8 Gill, Aisha 69–70 Global Corruption Barometer (GCB) 38 goals, impact 91 governance 50–1 government 16, 25, 37, 52, 90, 92, 99, 106–7 and child poverty 39, 91 Coalition 55 disability benefits 66 local 40 and migration 53 New Labour 32, 54–5, 85 and parenting and child behaviour 80 regenerative medicine 52 and welfare state 86, 94, 105 Guardian 58–9

F Falkner, Dr Robert 52 first generation of Social Policy scholars 32, 90–1, 95–6, 99

137

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

impact beneficiaries 78–81, 82, 104 impact goals 91 impact ladder 62 impact pathways 52, 63, 73, 74–5, 76, 89–90 income, basic 67 India 69–70 industry 1 inequalities, gender 71 innovation, policy 64–5, 104 institutional gaps 13 ‘interactive’ model 12–13 International Association of Chiefs of Police 69 International Institute for Democracy and Electoral Assistance (IDEA) 38 international organisations 33, 41, 70, 74, 75, 76, 86, 107 and gender inequalities 71 and protection from mass atrocities 67–8, 77–8 International Relations 26–7, 28, 29, 30, 35, 37, 46, 47, 72, 103–4 conceptual work 36, 43–4 description of sub-panel 109 impact agents 75–6, 81 panel membership 115–16 policy discourse 73 publications 42, 43 ratio of male to female sole/lead applicant 48 REF ranking 113 research centres 45 internet-polling methods 58 intimacy 40 Iraqi Kurdistan 69–70

H Haagh, Louise 67 Haas, PM 46 habits 52, 53 Halsey, AH 27, 33 Haux, T 63 Head, BW 11, 14, 16, 18, 50 health technologies 79–80 HEFCE 8–9, 22 Heinrich, CJ 18 homicide investigation 69 ‘honour’ based violence (HBV) 70 Hudson, J 26 human development 42 human rights 37, 38, 67–8

I ICS (impact case studies) 3, 7, 9, 22–34, 36, 38–40, 48, 51–60, 72–4 and commissioned research 44–5 and conceptual research 40–2 and impact beneficiaries 78–81, 82, 104 influence on practice 74 and REF ranking 113 by type of university 49–50 identity academic 97–8 collective 95–6 illicit drug policy 67 impact definition 22 research 63 impact agents 75–9, 81, 104

138

INDEX

Mead, LM 13–14 media 58–9 medicine, regenerative 51, 52 mentoring 95–6, 99, 100 ‘Migrapass’ 40 migration 40, 52, 53 Migration and Global Environmental Change: Future Challenges and Opportunities (MGEC) 53 military ethics 69 Modood, Professor Tariq 77 Mort, Maggie 79 Morton, S 62–3, 64 Munro, Eileen 57 Murder Investigation Manual 69 Muslims 77

J Just War Theory 69

K Keele, University of 31 Kent, University of 45 Khazragui, H 26 King’s College, London 42 knowledge 4, 70–1, 72–3, 88–9 knowledge brokers 17, 19, 82 ‘knowledge’ model 12 Kwiatkowski, R. 74

L labour market integration 40 Lancaster, University of 53 Landman, Professor Todd 38 Leeds, University of 31, 78 Levitas, Ruth 41 Lindblom, CE 2, 74–5 Living in Dignity in the 21st Century: Poverty, Human Rights and Democracy 67 local government 40 London 40 lone scholar 46, 47, 103 Loughborough, University of 45 LSE 45, 52

N nanotechnologies regulation 52 Newburn, Tim 58–9 new knowledge 4, 70–1, 72–3 New Labour government 32, 54–5, 85 Newman, J 14 new policies 3–4, 64–5, 73, 81 New Public Management 11

O older people disability benefits for 66 remote care for 79–80 online-survey sector 58 ONS (Office for National Statistics) 39 organised crime 68–9 Oxford, University of 56

M Mackenzie, Adrian 79 Manchester, University of 51, 71 McCargo, Duncan 78 McLean, Professor Iain 71

139

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

poverty 37, 39, 41, 65, 88–9, 91, 96 see also child poverty Poverty and Social Exclusion (PSE) survey, 1999 41 professionals, practice of 68–70 professors, social policy 31–3, 34, 83–100, 105–7 publications, dissemination of findings through 42–3 public engagement 58–9 Public Safety Canada 68

P panel membership 114–20 parenting programmes 80 pathways, impact 52, 63, 73, 74–5, 76, 89–90 Payne, Dr Clive 56 PEPFAR-USAID 80–1 police ‘stop and search’ practice 59 policies, new 3–4, 64–5, 73, 81 policy actors, academics engagement with 17–18, 39, 99, 102, 107 policy context 3 policy direction 4, 65–6, 72–3, 86–7 policy discourse 4, 66–8, 73, 74, 88 policy innovation 64–5, 104 policy intervention 73 see also new policies policymaking, and research 2, 7, 11–19, 103 policy practice 4, 72–3 political climate 85–6 political engagement 97–8 Politics 28, 29, 30, 35, 37, 46, 47, 48, 72, 103–4 British 26–7 conceptual work 36, 43–4, 59 description of sub-panel 109 and impact agents 75–6, 81 panel membership 115–16 policy discourse 73 publications 42, 43 REF ranking 113 research centres 45 Portugal 67 post-1992 universities 49–50, 103

R ranking, universities 113 REF2014 8–11, 19, 21–3, 28, 83–4, 85, 91, 101 critique of impact in 93–5 and governments 99 sub-panels 109–20 REF2021 19, 108 regenerative medicine 51, 52 regional expenditure data 71 regional funding 71 relational approach 16 relationships, intimate 39, 40 research nature of the underlying 3, 103 and policymaking 2, 7, 11–19, 103 research centres 45 ‘Research Contributions Framework’ (CAF) 62–3 research funding 43–5, 50 research impact 63 Research Selectivity Exercise 1, 2 research uptake 62–3 research use 63 Responsibility to Protect 78

140

INDEX

Social Policy Research Unit (SPRU), University of York 54–5 Social Work 28, 29, 30, 31, 35, 36–7, 46, 47, 48, 72 description of sub-panel 110–11 impact agents 75, 76 new policies 73, 81 panel membership 117–19 publications 43 REF ranking 113 research centres 45 research funding 43, 44 Sociology 27, 28, 29, 30, 37, 46, 47, 48, 72 description of sub-panel 111–12 impact agents 75, 76 knowledge brokers and advocates 82 panel membership 119–20 policy discourse 73 publications 42, 43 quantitative studies 36 REF ranking 113 research funding 43, 44, 45 South Africa 39, 80 South Wales 31 statistical confidentiality practices 51 stem cell research 51, 52 STEM subjects 25–6 Stern review 108 Stewart, E 62, 63 sub-panels for the REF2014 109–20 Sub-Saharan Africa 80–1 support, and trust 16–17

‘Rights of People Experiencing Poverty’ project 67 riots, 2011 58–9 Roberts, Celia 79 Rodin, David 69 Roehampton, University of 31, 40 Rose, H. 38 Russell Group universities 31, 49, 103

S Save the Children 81 second generation academics 32–3, 34, 90–3, 105–7 Serious Organised Crime Agency (SOCA) 68 sexualities 39, 40 Shove, Elizabeth 53 sickle cell 79 Smith, KE 15, 62, 63, 67–8, 104 Smith et al 8 social care services 81 social deprivation 41 social exclusion 41 social inclusion 40 Social Policy 27, 28, 29, 30, 35, 36, 46, 47, 48, 72 description of sub-panel 110–11 impact agents 75, 76 new policies 73, 81 panel membership 117–19 professors 31–3, 34, 83–100, 105–7 publications 42, 43 REF ranking 113 research centres 45 research funding 43, 44

141

DIMENSIONS OF IMPACT IN THE SOCIAL SCIENCES

internal research funding 50 relationship between research and policymaking 7 Russell Group 103 see also individual universities UoM (University of Manchester) 51, 71

T technical changes 50 telecare 79–80 Thailand 78 thalassaemia 79 timeliness 14–15 timing 92–3, 99, 105, 106, 107 Titmuss, Richard 32 Transparency International 38 trauma recovery 64 trust 16–17 two-communities model 13, 17

V Varese, Federico 68–9 violence against women (VAW) 69–70 voting behaviour 56

U UEA (University of East Anglia) 57 UK deprivation 39 Equality Performance Indicators 70–1 migration 40 Muslims 77 poverty 41 violence against women (VAW) 69–70 UNICEF 81 Universal Credit 54, 94 universities 2, 8, 29, 31, 49–50, 101, 113 cost of REF 10 employed professional writers 23 extent of individualisation 84, 99 and industry 1 influence of academics more visible to 105

W Wales, South 31 Watermeyer, R 23 Webster, Professor Andrew 51–2 Weiss, C 12, 13 welfare state 86, 94–5, 105–7 well-being 37, 39, 81, 88–9 Welsh, Professor 77–8 Western militaries 69 window of opportunity 107 women practice regarding in ECOWAS 42 violence against 69–70 working time 71 work–life balance 71

Y York, University of 39, 45, 54–5 youth, ECOWAS 42

142