Handbook on Risk and Need Assessment: Theory and Practice 9781138927766, 9781315682327

Citation preview

Risk and need assessment has been one of academic criminology’s biggest achievements and biggest disappointments. Although assessment tools are ubiquitous across correctional systems, they have fundamentally failed, to date, to alter the culture of correctional practice. In this urgent new volume, the most important and influential assessment researchers take stock of the successes, failures, and futures of the practice, exploring both the evolving science of risk prediction and the art of implementation. A most promising start to the new DCS Handbook Series. —Shadd Maruna, Professor of Criminology, University of Manchester, UK Accurate assessment of offender risks and needs is the bedrock of efforts to improve public safety. This handbook provides a critical foundation for advancing science and policy by illuminating the tremendous progress in assessment that has occurred. It is a must-read for anyone seeking to create a safer and more just society. —Daniel P. Mears, PhD, Mark C. Stafford Professor of Criminology, Florida State University College of Criminology and Criminal Justice, USA At virtually every stage of the justice system, new methods are being employed that enable decision-­ makers to use risk to the public as a criterion for justice system control.Yet even as these methods diversify, the empirical foundation for risk assessment remains a work-in-progress. Too little is known about the practical significance of risk as a core justice construct and the corresponding operational significance of risk assessment as a technique. This collection brings together superb studies of risk in the correctional system, both as an idea and as a practice. It is a welcome new contribution to our understanding of the most important development in the current generation of tools for the justice professions: risk assessment. —Todd R. Clear, University Professor of Criminal Justice, Rutgers University-Newark, USA The major strength of the Handbook on Risk and Need Assessment is that it provides researchers and practitioners with a comprehensive collection of chapters that helps chart the topic from its history to the implications for practice and policy. It is a must-have for anyone working or studying in the field of corrections. —Edward Latessa, Professor and Director, University of Cincinnati, USA One of the most intense activities by the many agencies responsible for managing accused or convicted offenders these days is assessment of their needs and of their treatment needs, especially for addiction or mental illness, or their risk in the community, whether that be on pre-trial release rather than bail, sentencing, and parole or probation release or recommitment decisions. This volume has pulled together a rich array of chapters from the wide variety of perspectives involved in assessing risk and needs from both methodological and implementation perspectives. —Alfred Blumstein, J. Erik Jonsson University Professor Emeritus, Heinz College, Carnegie Mellon University, USA

This page intentionally left blank

Handbook on Risk and Need Assessment

The Handbook on Risk and Need Assessment: Theory and Practice covers risk assessments instruments used in justice settings, regardless of whether it is for those confined in prisons or jails, on probation or parole, on pretrial status, at arrest, or for other key decision points in the justice system. As the United States begins to examine how to move away from ineffective, expensive, and unfair policies that grew out of crime control efforts aimed at mass criminalization, risk and needs assessment tools have emerged as practices to recalibrate decision-making. The ASC Division on Corrections & Sentencing Handbook Series will publish volumes on topics ranging from violence risk assessment to specialty courts for drug users, veterans, or the mentally ill. Each thematic volume focuses on a single topical issue that intersects with corrections and sentencing research. Faye S. Taxman, PhD, is a University Professor in the Criminology, Society and Law program at George Mason University in Fairfax, VA, and Director of its Center for Advancing Correctional Excellence. She is a well-regarded scholar and researcher who served as Chair from 2013–2015 of the American Society of Criminology’s Division on Corrections & Sentencing.

The ASC Division on Corrections & Sentencing Handbook Series Edited by Pamela K. Lattimore and John R. Hepburn Editorial Board: Alfred Blumstein, Carnegie Mellon University Todd Clear, Rutgers University Beth Huebner, University of Missouri, St. Louis Brian Johnson, University of Maryland Doris MacKenzie, Pennsylvania State University Shadd Maruna, Rutgers University Joan Petersilia, Stanford University Cassia Spohn, Arizona State University Susan Turner, University of California, Irvine Jeffery Ulmer, Pennsylvania State University Steve Van Dine, Ohio Department of Rehabilitation and Correction Christy Visher, University of Delaware The American Society of Criminology’s Division on Corrections & Sentencing sponsors a series of volumes published by Routledge on seminal and topical issues that span the fields of sentencing and corrections. The critical essays, reviews, and original research in each volume provide a comprehensive assessment of the current state of knowledge, contribute to public policy discussions, and identify future research directions. Each thematic volume focuses on a single topical issue that intersects with corrections and sentencing research. The contents are eclectic in regard to disciplinary foci, theoretical frameworks and perspectives, and research methodologies. Short Title: ASC Handbook Series 1. Handbook on Risk and Need Assessment Theory and Practice Taxman 2. Punishment Decisions Locations of Disparity Ulmer and Bradley

HANDBOOK ON RISK AND NEED ASSESSMENT Theory and Practice

Edited by Faye S. Taxman

MANAGING EDITOr AMY DEZEMBEr

First published 2017 by Routledge 711 Third Avenue, New York, NY 10017 and by Routledge 2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN Routledge is an imprint of the Taylor & Francis Group, an informa business © 2017 Taylor & Francis The right of the editor to be identified as the author of the editorial material, and of the authors for their individual chapters, has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging in Publication Data A catalog record for this book has been requested ISBN: 978-1-138-92776-6 (hbk) ISBN: 978-1-315-68232-7 (ebk) Typeset in BemboStd by codeMantra

CONTENTS

Notes on Contributors x Introduction xv 1 The Value and Importance of Risk and Need Assessment (RNA) in Corrections & Sentencing: An Overview of the Handbook 1 Faye S.Taxman with Amy Dezember PART I

History of RNA 21 2 Risk and Needs Assessment in Probation and Parole: The Persistent Gap Between Promise and Practice 23 William D. Burrell 3 The Research Director Perspective on the Design, Implementation, and Impact of Risk Assessment and Offender Classification Systems in USA Prisons: A National Survey 49 James M. Byrne and Amy Dezember PART II

Methodological Issues in Creating and Validating RNA 65 4 Static Risk Factors and Criminal Recidivism 67 Robert Brame 5 Accuracy of Risk Assessment in Corrections Population Management: Where’s the Value Added? 93 James Hess and Susan Turner

vii

Contents

6 Improving the Performance of Risk Assessments: A Case Study on the Prediction of Sexual Offending among Juvenile Offenders 114 KiDeuk Kim and Grant Duwe 7 Using Predictive Analytics and Machine Learning to Improve the Accuracy and Performance of Juvenile Justice Risk Assessment Instruments: The Florida Case Study 140 Ira M. Schwartz, Peter York, Mark Greenwald, Ana Ramos-Hernandez and Lisa Feeley 8 An Alternative Scientific Paradigm for Criminological Risk Assessment: Closed or Open Systems, or Both? 164 Tim Brennan PART III

Dynamic Risk Factors and Responsivity Toward Different Populations 191 9 Risk, Need, and Responsivity in a Criminal Lifestyle 193 Glenn D.Walters 10 Gender-Responsive Risk and Need Assessment: Implications for the Treatment of Justice-Involved Women 220 Emily J. Salisbury, Breanna Boppre and Bridget Kelly 11 Advancing Sexual Offender Risk Assessment: Standardized Risk Levels Based on Psychologically Meaningful Offender Characteristics 244 R. Karl Hanson and Guy Bourgon 12 Incorporating Procedural Justice and Legitimacy into the RNR Model to Improve Risk-Need Assessment 269 Katherine Ginsburg Kempany and Kimberly A. Kaiser 13 Adaption of Risk Tools to Employment Context 292 Garima Siwach and Shawn D. Bushway 14 Exploring How to Measure Criminogenic Needs: Five Instruments and No Real Answers 312 Brittney Via, Amy Dezember and Faye S.Taxman PART IV

RNA Implementation and Practice 331 15 Customizing Criminal Justice Assessments 333 Zachary Hamilton, Elizabeth Thompson Tollefsbol, Michael Campagna and Jacqueline van Wormer viii

Contents

16 Risk/Need Assessment Tools and the Criminal Justice Bureaucrat: Reconceptualizing the Frontline Practitioner 378 Joel Miller and Sarah Trocchio 17 Risky Needs: Risk Entangled Needs in Probation Supervision 406 Danielle S. Rudes, Jill Viglione and Kimberly S. Meyer PART V

Special Issues Regarding the Conceptualization for RNA 429 18 Purpose and Context Matters: Creating a Space for Meaningful Dialogues About Risk and Need 431 Kelly Hannah-Moffat 19 Human Rights, Risk and Need: The Right to Rehabilitation, and the Right to Fairness 447 Mary Rogan Index465

ix

CONTRIBUTORS

Breanna Boppre is a doctoral student in the Department of Criminal Justice at the University of Nevada, Las Vegas. Her current research interests involve intersections between gender and race in criminology and criminal justice as well as gender-responsive correctional programs and practices. Guy Bourgon is an Adjunct Professor at University of Ottawa and a researcher at the Corrections Research Public Safety Canada. He is a clinical psychologist specializing in correctional and criminal justice psychology. As co-lead for the Strategic Training Initiative in Community Supervision (STICS), an empirically supported and internationally recognized best practice model of community supervision, he is recognized for translating research into useful and practical concepts and skills. Robert Brame is a Professor in the Department of Criminology and Criminal Justice at the University of South Carolina. His research interests include criminal recidivism, linkages between demography and crime and criminal justice system involvement, and issues of crime measurement. Tim Brennan is probably best known for his book on The Social Psychology of Runaways (co-authored with Del Elliott and Dave Huizinga) and chapters on criminological classification in Criminal Justice Annual Reviews. He received the 2007 Warren-Palmer Award from the American Society of Criminology for contributions to offender classification and differential treatment. William D. Burrell is an independent corrections management consultant specializing in community corrections and evidence-based practices. Mr. Burrell served for nineteen years as chief of adult probation services for the New Jersey state court system. Shawn D. Bushway is a Professor of Public Policy and Management and a Professor of Criminal Justice at the University at Albany (SUNY). He is an expert on employer use of criminal background checks for hiring. James M. Byrne is a Professor at the School of Criminology and Justice Studies at the University of Massachusetts, Lowell. His work covers a range of corrections and sentencing including x

Contributors

global prison population trends, evidence-based sentencing and corrections, offender change, prison violence, risk classification, the effectiveness of crime control technology, and the community context of crime. He has been awarded many honors including Distinguished Scholar Award and the Marguerite Q. Warren and Ted Palmer Differential Intervention Award from the American Society of Criminology’s Division on Corrections & Sentencing. He is the Editor of the journal, Victims and Offenders: An International Journal of Evidence-Based Research, Policy and Practices. Michael Campagna is a Senior Research Associate for the Washington State Institute for Criminal Justice (WSICJ) and a doctoral candidate at Washington State University. He received his master’s degree from Indiana University of Pennsylvania in 2009 and his bachelor’s degree from Penn State in 2006. His most recent publication appeared in Criminal Justice and Behavior and Law and Criminal Justice and his research interests include reentry, risk assessment, criminology, and mental health. Amy Dezember is a doctoral student in Criminology, Law and Society  at George Mason University. She  worked at the Center for Advancing Correctional Excellence (ACE!) with Dr. Taxman, focusing on risk and need assessment tools. Her primary research interests include juvenile transfer, court processes, and indigent defense. Grant Duwe is Research Director for the Minnesota Department of Corrections (MnDOC), where he conducts program evaluations, develops risk and needs assessments, and forecasts the state’s prison population. He has also been named Visiting Fellow at the Bureau of Justice Statistics. He is the recipient of the Donal MacNamara Outstanding Publication Award by the Academy of Criminal Justice Sciences and the inaugural Research Practitioner Award by the American Society of Criminology’s Division on Corrections & Sentencing. Lisa Feeley is a Research Associate at ICF where she manages state-wide needs assessments to identify and analyze gaps in crime victim service provision. Her research interests cover a broad spectrum of issues including victim services, sexual assault response and policy, juvenile justice, and human trafficking. Mark A. Greenwald is the DJJ Director of Research & Data Integrity for the Florida Department of Juvenile Justice. He is also doctoral student in the College of Criminology and Criminal Justice at Florida State University. His research interests include juvenile justice, delinquency, and crime in schools. In 2008, he received the “Distinguished Alumni of Colleges” academic award from Florida Atlantic University. Zachary Hamilton, Ph.D., is an Associate Professor of Criminal Justice and Criminology at Washington State University and Director of the Washington State Institute for Criminal ­Justice. He received his Ph.D. in criminology and criminal justice from Rutgers University in 2010. His research interests include corrections, offender reentry, assessment, substance abuse treatment, and quantitative methods. Kelly Hannah-Moffat is a Professor in Sociology/Criminology & Sociolegal Studies and Vice President of the University of Toronto. She conducts interdisciplinary research on risk, human rights, gendered justice, punishment and marginalized and diverse populations. Her work on risk, gender and punishment focuses on how variables such as gender and race interact with seemingly objective assessment tools, the experiences of the assessors and the institutional operationalization of policy reforms. xi

Contributors

R. Karl Hanson, Ph.D., C.Psych, is a Manager, Research Division, at Public Safety Canada, and Adjunct Professor in the Psychology Department of Carleton University, Ottawa, Ontario, and Ryerson University, Toronto, Ontario. His research focusses on the assessment and treatment of sexual offenders. He is co-author of several widely used sexual offender risk tools, including Static-99R, Static-2002R and STABLE-2007. Jim Hess is a Senior Statistician and Analyst for the Center for Evidence-Based Corrections. He has a Ph.D. in Social Science and has worked in epidemiology, public health, and anthropology, publishing on diverse topics including migration, economic development, Pacific Islander use of medical practitioners, and the epidemiology of cancer. Kimberly Kaiser is an Assistant Professor of Legal Studies at the University of Mississippi. She has published research in the areas of judicial and victim decision-making, theories of procedural justice and legal socialization, and specialized court programs. Bridget Kelly is a doctoral student of Criminology and Criminal Justice at the University of Nevada, Las Vegas. Her research interests include risk assessment and community corrections. Katherine Ginsburg Kempany is a Doctoral Candidate in the School of Criminology and Criminal Justice at Arizona State University. Her research interests include influences on and responses to inmate misconduct, inmate behavior management strategies, and discretionary decision-making in community and institutional corrections. KiDeuk Kim is a Senior Research Associate in the Urban Institute’s Justice Policy Center, where he examines an array of issues related to crime and criminal justice policy. He is currently leading an effort to develop resources for the Risk Assessment Clearinghouse, a national repository of evidence-based information on risk and needs assessment, funded by the Bureau of Justice Assistance. Kimberly S. Meyer is a Graduate Research Assistant at the Center for Advancing Correctional Excellence (ACE!) and doctoral student in Criminology, Law and Society at George Mason University. Her primary research interests include juvenile corrections, implementation of evidence-based practices (EBPs), and severe, life, and long-term sentencing.  Joel Miller is an Associate Professor at Rutgers University’s School of Criminal Justice. His research has examined a range of applied policy research topics, including police tactics and accountability, juvenile justice, risk assessment, and crime prevention. It focuses in particular on strategies for improving the effectiveness of and public confidence in criminal justice agencies and policies. Ana Ramos-Hernandez is Chief Data Whisperer for Algorythm. Mary Rogan is an Associate Professor of Law at Trinity College, Dublin, Ireland, and a barrister (attorney). She is the Principal Investigator on ‘Prisons: the rule of law, accountability and rights’, funded by the European Research Council. She is the author of Prison Law (Bloomsbury 2014) and Prison Policy in Ireland: Politics, Penal-Welfarism and Political Imprisonment (Routledge, 2011), and is a member of the International Penal and Penitentiary Foundation. Danielle S. Rudes is an Associate Professor of Criminology, Law and Society and the Deputy Director of the Center for Advancing Correctional Excellence (ACE!) at George xii

Contributors

Mason University. Dr. Rudes is a qualitative researcher whose methods include ethnographic observation, interviews, and focus groups. She is recognized for her work examining how social control organizations and their middle management and street-level workers understand, negotiate, and at times, resist change. Emily J. Salisbury is an Associate Professor of Criminal Justice at the University of Nevada, Las Vegas and the editor-in-chief of Criminal Justice and Behavior. Dr. Salisbury’s current research is focused on reformulating the principles of effective correctional intervention for justice-­involved women both conceptually and in practice for women in the criminal justice system. Ira M. Schwartz is Consultant and Advisor to Algorhythm, and former Provost, Temple University and Dean of the School of Social Policy and Practice, University of Pennsylvania. Schwartz also served as Administrator, Office of Juvenile Justice and Delinquency Prevention, U.S. Department of Justice. Garima Siwach is a Ph.D. candidate at the Department of Economics, University at Albany, SUNY. Her research focuses on using applied econometrics to study the interactions between criminal justice involvement and labor market outcomes. Faye S. Taxman is a University Professor in the Criminology, Law and Society Department and Director of the Center for Advancing Correctional Excellence at George Mason University. She has active “laboratories” with the Maryland Department of Public Safety and Correctional Services and developed the RNR Simulation Tool (www.gmuace.org/tools). The American Society of Criminology’s Division of Sentencing & Corrections has recognized her as Distinguished Scholar twice as well as the Rita Warren and Ted Palmer Differential Intervention Treatment award. Elizabeth Thompson Tollefsbol, M.A., is a Senior Research Associate for the Washington State Institute for Criminal Justice (WSICJ) and a visiting professor at Gonzaga University. She received her master’s degree from Washington State University and her bachelor’s degree from Fordham University. Her research interests include risk assessment, program effectiveness, gender responsivity, and reentry. Sarah Trocchio is a Ph.D. student at the Rutgers School of Criminal Justice. Her research interests include policing in urban areas, race and ethnicity, and front-line decision making. Susan Turner is a Professor in the Department of Criminology, Law and Society at the University of California, Irvine and Director of the Center for Evidence-Based Corrections. Dr. Turner’s areas of expertise include the design and implementation of randomized field ­experiments and research collaborations with state and local justice agencies. Jacqueline van Wormer, Ph.D., is the criminal justice administrator for Spokane County, Washington. Her research focuses on implementation challenges in corrections, juvenile justice trends, measuring the impact of swift and certain probation methods, and the validation of risk/ need/responsivity tools. Brittney Via is a Research Specialist in the Office of Research and Strategic Planning (ORSP) at the West Virginia Division of Justice and Community Services (DJCS). She has an M.A. in Criminology, Law and Society from George Mason University and Randolph College with a B.A. in Sociology. xiii

Contributors

Jill Viglione is an Assistant Professor at the University of Texas at San Antonio. She specializes in applied research focusing on evidence-based practice implementation, organizational change, and decision making within correctional settings. Glenn D. Walters is an Associate Professor in the Department of Criminal Justice at Kutztown University (Pennsylvania). He has worked as a correctional psychologist at the United States Penitentiary in Leavenworth (Kansas), the Federal Correctional Institution in Fairton (New Jersey), and the Federal Correctional Institution in Minersville (Pennsylvania). His current research interests include criminal thinking, causal mediation analysis, and development of an overarching psychological theory of crime. Peter York is Founder and President of Algorhythm.

xiv

INTrODUcTION

It is with pleasure that we introduce this annual handbook series as a forum for compelling and integrative research on topics that crosscut research, policy, and practice in adult and juvenile sentencing and corrections. This handbook series reflects a collaboration between the Division on Corrections & Sentencing of the American Society of Criminology and Routledge Press. The Division was established as a constituent unit of ASC in 1999 with multiple objectives, including “to facilitate and encourage research on corrections and sentencing for adults and juveniles,” “to facilitate and encourage research pertaining to sentencing, rehabilitation, punishment, community and institutional corrections, diversionary programs and alternatives,” and “to facilitate and encourage interaction and dissemination of research among ASC members, practitioners, funding ­agencies, policy-making bodies, corrections and sentencing organizations, and other relevant groups.” This Handbook provides one means to address these objectives. We are at an interesting juncture in the United States with respect to criminal and juvenile justice reform and the implications of reform for sentencing and correctional policy. Following decades of growth in correctional populations (both institutional and community supervised), we are currently witnessing calls from multiple quarters to reduce “mass incarceration” through reforms focused on multiple avenues including decriminalization, diversion, and decarceration through either alternatives or reduced sentencing. These calls for reforms provide an excellent opportunity for thoughtful scholars, policy makers, and practitioners to examine current practices, to take advantage of natural experiments associated with changes in policy and practice, and to explore alternatives that may change the face of sentencing and corrections in the United States. Whether we are witnessing a true turning point that will return the size of criminal justice populations in the United States to historical levels or simply a leveling off of growth remains to be seen; however, there does appear to be a national appetite for moving beyond the policies and practices that derived from the “War on Crime” and the “War on Drugs.” Of course, this Handbook is intended to reach beyond the shores of the United States.We anticipate comparative works, as well as international research and evaluation, as major contributions to the series. The Handbook is an alternative venue for peer-reviewed publication beyond the traditional journals. Over the past 15 years, the Division has periodically addressed the issue of supporting a publication that would reflect the objectives of the Division. After exploring the possibility of sponsoring a new journal, the Division decided that a Handbook Series would provide a forum for longer “think pieces” and critical examinations of emerging issues in corrections and sentencing,

xv

Introduction

and their interstices. In particular, the Handbook is intended to provide scholars with an opportunity to write to specific cross-cutting themes with less concern about page limits and format. We are pleased to introduce this inaugural volume focused on risk and needs assessment and look forward to upcoming volumes that will address topics related to disparity and collateral consequences. We would like to thank the wonderful members of our editorial board for their support and contributions, the editorial and marketing staff of Routledge Press for their support, and the editor of our inaugural volume, Professor Faye Taxman, for her efforts in assembling this important collection.We hope you find this volume informative and useful and that you will consider contributing to future volumes.

xvi

1 ThE VALUE AND IMpOrTANcE OF RISK AND NEED ASSESSMENT (RNA) IN COrrEcTIONS & SENTENcING An Overview of the Handbook Faye S. Taxman with Amy Dezember1

Missouri’s Chief Justice Ray Price (2010) in his State of the Judiciary speech: “There is a b­ etter way. We need to move from anger-based sentencing that ignores cost and effectiveness to ­evidence-based sentencing that focuses on results—sentencing that assesses each offender’s risk and then fits that offender with the cheapest and most effective rehabilitation that he or she needs.” (cited in Casey,Warren & Elek, 2012: pg. 12) Policymakers’ need to know the subsequent strategies for public safety and recidivism reduction might begin with a simple question: Do risk assessment instruments reliably predict recidivism? The short answer, according to years and volumes of research, is resoundingly: yes. But we must be mindful of what saying yes may mean. Adoption of a risk assessment tool goes hand-in-hand with fundamentally altering approaches to reentry and correctional management, supervision, services, and more broadly criminal justice practice. Ultimately, the process of implementing risk assessments within an agency should consist of more than simply adding a tool to the agency portfolio; it should result in a shift of corrections culture, practices, and policies. (cited in Desmarais & Singh, 2013) The Public Safety Performance Project at Pew put out an issue brief titled: “Risk/Needs Assessment 101: Science Reveals New Tools to Manage Offenders” (The Pew Center on the States, 2011)

Most advancements in managing individuals in the criminal justice system begin with a discussion of the need for the adoption of a standardized risk and need assessment (RNA) tool. ­Standardized, valid tools are research-based in that they are built using statistical methods to predict desired (or undesired) outcomes. RNA tools are generally recommended at all decision points along the criminal justice system—from booking to pretrial release to sentencing to release from prison or jail to services. Standardized RNA tools, in theory, offer a science-based approach to regulate decision-making to avoid or minimize biases, decrease unnecessary discretion, improve proper use of resources, and/or increase fairness. As noted above in the various headlines of recent policy pieces, RNA tools are considered the panacea to better practice. Great promises are tied to the use of RNA tools, primarily better performance of the justice system at the system level to deliver 1

Faye S. Taxman with Amy Dezember

fairness and justice, at the program level to assign individuals to appropriate controls and programs, and at the individual level to improve outcomes.The rapid growth and expansion in risk assessment tools—in various domains of the justice system—illustrates a growing appreciation for integrating science into practice in order to improve operational practices. The Division of Corrections and Sentencing of the American Society of Criminology is devoting the first edition of our Handbook series to this timely topic of RNA tools. The interest in RNA tools is dear to the hearts of practitioners as well as researchers and academics. The rapid expansion of RNA tools in various justice settings—generating a growth in both public domain and proprietary tools, even for special offenders—requires more attention to better understand how RNA tools are developed, implemented, and impact justice. In fact, given the current body of tools being employed for various goals, it is a worthy endeavor to assess the “state of the art”. This review of both theory and policy is devoted to critically analyzing the issues surrounding the methods to develop RNA and then how this RNA tool is used to fulfill the grand vision of having more precision in decision-making among justice actors as part of the process of diagnosis, prediction, and linkages to better outcomes. We are grateful that members of the American Society of Criminology were willing to critically assess the state of RNA, and to help identify a research agenda that advances the field. Given that RNA is tied to the linkage of science influencing practice, the analysis includes a research agenda to advance the field of RNA tool development and utilization, which is needed to garner better implementation and utilization of RNA tools.

The Generations of RNA Risk and need assessment tools (RNAs) represent a research-based application relevant to the operations of many justice agencies. RNAs are standardized tools that apply scientific principles to develop and test tools that are presumably valuable to the field. The onset of the movement to build and use standardized risk and need assessment tools highlights the need to improve decision-­ making in the justice system as compared to unstructured, or even semi-structured, interviews. The first generation of tools involves the use of justice actors to make decisions using information that is available on an individual. In this first generation, the approach was to rely on clinical and professional judgment that does not have explicit or objective scoring rules (Brennan, Dieterich, & Ehret, 2009). The first generation essentially dominated institutional and community corrections for several decades, and remains preferred by many decision makers (Boothby & Clements, 2000; Wormith, 2001). Unstructured or semi-structured interviews depend on the quality of the staff and their ability to use the appropriate information in making a decision. The general concerns with the interview approach is that it is characterized by excessive subjectivity, inconsistency, bias and potential stereotyping, which raises the issues of vulnerability in decision-making (Brennan, 1987; Grove & Meehl, 1996; Hastie & Dawes, 2001). Individual actors can select the information they consider to be relevant, which may vary across actors, and the information they find relevant may not be empirically related to individual outcomes. In general, the use of unstructured information relies on the skill set of the individual actor, which can create the possibility of bias or fluctuations in discretionary decision-making. The second generation of RNA tools represents the beginning of structuring information in the tools based on the criminal history and case file.The selected information items draw from research findings on the factors that are linked to the outcome, which is generally recidivism (in whatever form it is measured such as arrest, reconviction, incarceration, etc.). The selection of the items requires statistical analyses to identify those factors that predict the outcomes as well as the weight given to any item. Second generation tools rely on risk prediction and are usually characterized by their brevity and efficiency (Brennan, Dieterich, & Ehret, 2009). The relatively simple point scales associated with the second generation tools were surprisingly effective in terms of predictive validity and generally outperformed professional judgment or the opinions of trained experts 2

The Value and Importance of RNA

(Dawes, 1979). Dating back to the 1920s, the idea of a parole release decision tool was conceived to reconcile a person’s past criminal history with the decision to be released early from prison. Risk assessments transformed into a tool to help parole examiners sort through information about the individual and identify the relevant factors associated with reincarceration or other recidivism measures. The process of tallying up past behavior resulted in the creation of the second generation of risk assessment tools, which in this case assisted in providing better guidance to the Parole Board on an individual’s likelihood of having further involvement with the justice system through systematic identification of factors related to “failure”. The emphasis on using past information to inform present choices was recognized as a means to facilitate difficult decisions, and to reduce the risks associated with making that decision. Peter Hoffman, a scientist at the U.S. Parole Commission, developed the salient factor instrument for parole supervision as another example of using key factors that predict success in a standardized tool to collect information about individuals (Hoffman & Beck, 1997). The process of using statistics to identify the key set of factors related to success and failure was considered an advancement for the field. In the mid-1960s, the Vera Institute introduced risk assessment tools for bail decisions, a landmark event in the risk assessment field given the use of the tool at an earlier stage in the justice system decision-making, and the components of the tool. The Vera Institute’s tool expanded the list of relevant information in the instrument to include factors related to stability (i.e. housing, employment, etc.) some of which are dynamic risk factors and some are indicators of social and community stability. These dynamic risk factors were linked to the outcome of failure during the pretrial period either in terms of failure to appear for court or rearrest (Mamalian, 2011). Since the Vera Institute tool included criminal history and stability factors in the community, this commenced the beginning of the third generation of RNA tools with the added information on needs that affect outcomes. Third generation tools introduced a more explicit, empirically based, theory guided approach, with a broader selection of dynamic factors that were more sensitive to change (Brennan, Dieterich, & Ehret, 2009). By incorporating a host of need factors (i.e. those related to recidivism), third gene­ration tools became more widely used to assess offenders’ risk for recidivism and their treatment needs (Campbell, French, & Gendreau, 2009). Over several decades, the use of standardized decision making tools slowly crept into existence in justice organizations with scientists and academics working on tools for various uses. The famous Wisconsin Risk and Needs Assessment (WRA) was developed in the late 1970’s for probation and parole agencies and featured items that used both static risk factors and dynamic risk (need) factors. The WRA concept generated an interest in RNA tools being used to support resource allocation decisions (see O’Leary & Clear, 1984). The resource allocation decision is to assign higher risk/need offenders with more frequent supervision and/or services to prevent recidivism. The notion that the risk score could be used to guide the level and types of supervision was born. Andrews, Bonta, and Wormith (2006) then developed the Level of Service Inventory (LSI) in the 1980’s and expanded the number of domains in the RNA tool. The LSI instrument includes a criminal history domain along with domains related to substance use, accommodation, attitudes and orientation, employment, education, peers, and so on. Beginning in the 1980’s, the RNA emerged as a dominant theme for institutional and community corrections, which contributed to an expanding number of tools in the public domain and proprietary tools. As the use of the tools expanded, so did the different types of predicted behavior (i.e., tools for specialized populations such as sex offenders, driving while intoxicated, females, violence, are a few examples). The expansion in the number and types of tools, as well as the different applications, raises a number of unanswered questions. Frequent concerns are whether the added items improve the predictive validity of the tool, whether the added items foster improved decisions, and whether the elements of the tool are responsive to the needs of the individual.There is also the question of whether proprietary tools outperform public domain tools, or increase the capacity of the system to better use the tools. In many ways, RNA tools continue to evolve and a greater emphasis is on increasing the complexity of the tools. 3

Faye S. Taxman with Amy Dezember

Don Andrews, James Bonta, and Steve Wormith (2006) crafted the need for a fourth generation of RNA tools to advance decision-making in the area of case management. Case management is different than typical recidivism-related outcomes in that the goal is to identify the factors that are dynamic and amendable to change—factors that should be targeted through treatment, programs, or controls to reduce recidivism. Essentially, the same factors that are used to predict recidivism are then considered in terms of their ability to define malleable behavior that can be altered as a result of programming and services. Some newer instruments expanded the third generation tools to highlight the greatest area of dynamic risk/needs or added items to the instruments. The unanswered questions about the fourth generation tools are: 1) what are the predicted outcomes, and 2) do the tools facilitate the ability to identify the dynamic needs for programming and service purposes? A new movement is emerging in the field to use machine learning tools. Machine learning tools use “big data” with advanced statistical methods to calculate a predicted value.The algorithms are used with an assortment of criminal histories, prior justice involvement, and other data that are readily available through management information systems maintained by the justice agencies. The advantage of the machine learning algorithms is their ability to calculate the formula without having an individual (justice actor) complete an assessment tool, therefore removing the potential for human errors that come from administering an instrument. The machine learning models primarily focus on the static risk items since criminal history is maintained in automated records. In some way, this focus only on risk factors seems to revert back to second generation tools but is justified on the ease of administering the tool. Of course, many questions remain about these tools regarding whether they increase predictive accuracy, provide for greater utilization by staff and justice actors, and facilitate how agencies can accommodate the dynamic needs into risk assessment processes that have prefilled risk scores. The generations of RNA illustrate the degree to which science has influenced the development of decision support tools for the field as well as advanced criteria for key decisions at various points in the justice system. On the surface, it appears that the evolution of risk and need assessment has Table 1.1  Generations of Risk and Need Assessment Tools Generation

Types of Variables

Methods

Decision Points

First—subjective assessment Second—risk factors

Case file information

Interviews and judgment

Third—risk and need factors

Key variables assumed related to recidivism or failure to appear (justice outcomes) Key variables related to recidivism or failure to appear Key variables to identify criminogenic factors that are amendable to treatment “Big Data” Cannot integrate information that is not in a database

Traditional justice decisions Release from prison, level of supervision, or pretrial supervision Release decisions Pretrial supervision Supervision Level

Fourth—risk and need factors for case management

Fifth—machine learning

List of key variables Scoring sheet Linked to criminal history Regression Scoring sheet Regression

Scoring sheet Regression

Case management Release decisions Pretrial supervision Supervision Level

Random forest models Algorithms

Release decisions Pretrial supervision Supervision Level

4

The Value and Importance of RNA

been logical and progressive (see Table 1.1 above), and that this strategy influences practice going back nearly 100 years. But this is far from true—the emergence of risk and need assessment tools in the practice of criminal justice is actually slow and grinding. It is only within the past five years that there has been an upsurge in the use of risk and need assessment tools by justice agencies, now that the tools are classified as an “evidence-based practice” for most justice sectors. There is a push for the use of the tools, with the federal government embracing the use of the tools in all sectors of the system, foundations promoting the use of the tools, and state legislatures authorizing the required use of the tools to advance justice practices. The major foundations that are vested in reforming the justice system—Pew Public Safety Performance and Arnold Foundation—embrace risk and need assessment tools in all aspects of the justice system. In fact, there has been a proliferation of monographs, brief reports, and webinars all devoted to the “wonders” of using risk and need assessment tools in various segments of the justice system and by most justice actors. The reliance on risk and need assessment tools has contributed to an increase in the ­complexity of the tools, with specialized tools for various types of offenders such as sex offenders, young offenders, women, and those with mental illness. This growth is overwhelming and there is now a consensus that RNA tools are important to the business of justice. However, like most practices, there are increasing questions regarding the 1) design, development, and validation of RNA tools; and 2) their use in practice.These are not mutually exclusive categories; rather they are overlapping in that the practice affects the design and the design affects the practice.

The Science Embedded in RNA The labeling of the “generation of tools” corresponds to advances in theory behind the risk and need assessment tools as well as the expansion of the potential application of the tools. Implicit is the research foundation for each generation, namely: 1) the desired outcome can be predicted and the tools improve decision-making; 2) the items included in the risk and need assessment tools are believed to be based on findings from research studies, and therefore the items themselves are valid; and, 3) the statistical methods used in creating the tools are appropriate and validate that the tools are better than clinical judgment (first generation). While “risk and need assessment tools” are considered “scientifically valid”, there are a number of issues associated with RNA tools that may enhance or diminish their intended uses and the validity of the prediction. Some of the issues are related to how the tools are used in practice. Essentially, the science of risk and need assessment tools is an art of decision-making but the categorization that the tools are “evidence-based” increases the premium on the application. Can the desired outcome for the risk and need assessment tool be predicted? The initial premise for the risk assessment tool was to predict recidivism, either in terms of rearrest, reconviction, or reincarceration. That is, the goal was to improve clinical judgment regarding whether a person will return to the justice system, and the “guesstimating” of key factors that are related to recidivism or continued criminal activity. As the third generation tools arrived on the scene, the predicted outcome changed, and the goals, from the tools. Instead of focusing on predicting recidivism, the tools had to be tailored to the stage of the justice system being administered and the outcomes relevant at that stage: 1) bail decisions are focused on failure to attend in court for adjudication, 2) pre-trial detention decisions require calculating public safety risk and whether a person’s criminal history was sufficient to be concerned about future offending, 3) judges are concerned with whether the risk and needs should factor in sentencing decisions about the need for more restrictions or controls, and, 4) probation/parole/correctional case managers tailor the supervision plans to match the risk and needs of the offenders as an outcome. Most decisions are not about prediction of recidivism, but rather are decisions about fairness, justice, public safety, and allocation of resources to achieve public safety goals. Each represents different outcomes, and in some cases the stated outcomes may be mediators or moderators of recidivism rather than a direct outcome themselves. The nature of 5

Faye S. Taxman with Amy Dezember

the desired outcome is tied to both expectations and outcomes, but the outcome may be conditioned on the purpose of the decision and how it can be used. The underlying question here is whether the RNA tool is valid and accurate and facilitates more consistent, unbiased decisions across the actors making the decisions. Another major theme in the RNA literature is that tools are tied to resource allocations within the justice system, meaning that the results from the assessment are used to specify which indivi­ duals need more intensive controls and/or treatments. The goal of the tool is then to assist in managing the population by targeting appropriate levels of resources based on risk level. This presumes that the targeted resources of controls and/or treatments will likely be able to suppress criminal activity for the higher risk offenders. It assumes that the enhanced resources are not iatrogenic, which has been a theme in studies of intensive supervision (Petersilia & Turner, 1993; Taxman, 2002) and incarceration (Nagin & Cullen, 2013). A similar logic exists for the underlying concept of fourth generation tools, whereby the goal is to identify more serious, dynamic risk factors (needs) that, if addressed, will reduce the person’s risk of recidivating. That is, the success of the case management goal depends on whether the targeted programs and/or controls are effective in reducing recidivism as shown in the logic model in Figure 1.1. If the goal of the RNA tool is to predict recidivism (or appearance in court), but is used for other purposes, then there is a need to test the relationship between the individual level factors and recidivism to assess whether the items can predict who will do better with more or less resources. This has not been tested, and if we are going to be comfortable using the tools in this manner, then it will need to be examined and assessed. It could be that the issue is not one of predictive validity but rather sound implementation of the policies that are associated with using RNA tools to allocate resources, case manage, and provide protections to reduce the liabilities for making decisions about how to manage the individuals including how to prosecute, sentence, monitor in the community, and release on parole.The question then is not whether the RNA is valid or accurate in predicting outcomes, but rather whether the policy is accurate. For example, in studies of graduated sanction grids (which are premised on using modulated responses based on the severity of the behavior), most findings demonstrate that the grids are not well implemented or utilized by the probation/parole staff, and therefore it is impossible to test their effectiveness in changing behavior (Turner et al., 2012; Steiner, Travis, & Makarios, 2011). The question here is about policy implementation and not the efficiency of the tools. Under these circumstances, the unanswered issue should focus on understanding why the RNA is being infrequently or improperly used, not necessarily how it impacts the outcomes of the individuals under justice control. In exploring this further, the case management goal is to identify the dynamic risk factors (needs) that should be addressed to reduce recidivism. This presumes that the treatments or controls to address dynamic factors will result in a change in outcomes, or that the dynamic risk factors can be impacted through programming. Under the resource allocation framework, resources

Risk and Need Assessment Tool Results

Vary Controls Programs Services

Outcome Recidivisim

Policies

Figure 1.1  Logic Model of RNA and Various Outcomes

6

The Value and Importance of RNA

should be allocated to the higher risk offenders as a means of distributing scarce resources and ­controlling criminal conduct (specific deterrence) through “intervention” of treatment and/or controls. For example, in the intensive supervision experiments that intensified contact levels, use of drug testing, and other services for higher risk offenders, it was found that the intensive controls and/or services did not result in improved outcomes (Petersilia & Turner, 1993; Taxman, 2002). The ­failure was not in the accuracy of the risk prediction but rather in whether the RNA tool results were used to determine who needed higher levels of control and whether the interventions were structured to reduce recidivism. This is the challenge in evaluating whether RNA tools are useful to decision makers since the results are often intertwined with the policies or practices pursued. Thus, caution must be taken in examining how RNA tools assist agencies in various routine tasks such as classification, assignment to programs and services, decisions about sanctions, and targeting individuals for more controls—all of which are other outcomes that are not necessarily the same as the predictive validity of the tools overall. Are the items in the tools valid? The scientific foundation for the RNA is that the tools can predict certain outcomes, such as recidivism. The expectation is that specific items included in the tools are related to the desired outcome and should contribute to this prediction of the desired outcome. It is the responsibility of researchers to identify the variables that predict the outcomes, to ensure that the variables are measured correctly, and to make sure that each item on an RNA tool adds to the value of the prediction. This is no small task. For example, static risk instruments favor criminal history as the strongest predictor of recidivisim (Gendreau, Little, & Goggin, 1996). Typically, criminal history includes a simple account of all adult arrests (or convictions) to a count of felony arrests (or convictions). Or, the instrument can examine arrests for particular offenses such as property offenses, personal crimes, violent crimes, drug crimes, and so on (which is infrequently done). The number and type of items included in the instrument varies across different risk and need assessment tools. While one study examined which areas had the strongest predictor domains (Gendreau, Little, & Goggin, 1996), the presumption is that the research has converged on a set of factors, with specific measures, that are related to recidivism. But, given the variety of factors used in different RNA tools and the different measures, it is unclear as to how much the science has agreed to a core set of measures. This unaddressed question needs immediate attention to both solidify the scientific knowledge base and ease implementation in the field. How factors are measured matters, particularly in articulating the research findings to practitioners who want to know, with confidence, that there is scientific agreement about the key measures. A few ­systematic reviews or meta-analyses exist that examine across the RNA instruments, particularly the predictive validity of different domains and measures.While these studies validate the tools’ overall ability to predict, the studies do not explore other uses of the tools or the factors that should be included in the assessment tools. The validity of the RNA tool is one question while the other question is whether the users of the RNA tool feel confident in using the results.A recent observational study found that some practitioners in one agency may not necessarily agree with the items in a proprietary RNA tool as being “valid” or even useful in their decision-making (Viglione, Rudes, & Taxman, 2015). Part of the doubt comes from the tool’s inattention to high “stake” offenses such as sex offenses, gun crimes, domestic violence, or violence.The other part of the doubt is that the officers do not understand the main domains of the tools (such as criminal associates, criminal values, criminal lifestyle, etc.) and therefore are doubtful that these attributes add value to predicting risk. Officers also expressed concern that the measures of criminal history tend to treat all crimes the same without prioritizing more serious criminal behavior. Additionally, how an RNA tool is used, as a screener or for classification, can lead to variation in the predictive validity of the tool (Fazel, Singh, Doll, & Grann, 2012). Finally, the RNA items do not examine time between events or change in type of criminal activity—factors that are important to practitioners. The validity of the dynamic risk factors is an area that needs further attention and study. Most studies have generally found that adding the dynamic risk factors adds little to the predictive 7

Reviewed 147 studies and found that for all outcome measures, empirically derived actuarial measures were significantly more accurate than unstructured professional judgments. Also, tools designed to measure general recidivism were most successful in predicting any recidivism, violent recidivism, and sexual recidivism.

Level of Service Inventory Revised (LSI-R)

Does the LSI-R have predictive validity when assessing female recidivism? (Smith, Cullen, & Latessa, 2009) Are empirically based tools, structured professional judgments, and clinical interviews effective in predicting outcomes for sexual, violent, or any recidivism among sex offenders? (Hanson & Morton-Bourgon, 2009)

Static-99; Minnesota Sex Offender Screening Tool Revised (MnSOST-R); Level of Service Inventory Revised (LSI-R);Violence Risk Appraisal Guide (VRAG); Sex Offender Risk Appraisal Guide (SORAG); General Statistical Information on Recidivism (SIR); Sexual Violence Risk 20 (SVR-20); Juvenile Sex Offender Assessment Protocol (J-SOAP); Structured Assessment of Risk & Need; Historical Clinical Risk Management 20 (HCR-20); Risk for Sexual Violence Protocol (RSVP); Estimate of Risk of Adolescent Sexual Offense Recidivism (ERASOR) Which risk instruments and psychological Historical Clinical Risk Management 20 measures are most able to predict general (HCR-20); Level of Service Inventory Revised (primarily nonsexual) violent recidivism (LSI-R); Violence Risk Appraisal Guide (VRAG); General Statistical Information on and institutional violence in adults? Recidivism (SIR); Psychopathy Checklist (Campbell, French, & Gendreau, 2009) Revised (PCL-R)

Examined 88 studies and found that third-generation instruments predict recidivism better than second-generation instruments; and instruments that include dynamic risk factors predict violent recidivism more accurately than instruments that only include static risk factors. When predicting institutional violence, second-generation instruments and static risk factors were the strongest predictors of institutional violence. Little variation was found among the mean effect sizes of common actuarial or structured risk instruments.

Reviewed 131 studies and found that dynamic predictor (r = .15) domains perform at least as well as static domains (r = .12). The strongest static predictor domain is criminal history. The strongest dynamic predictor domains are criminal companions, interpersonal conflict, substance abuse, and antisocial personality. The LSI-R produced the highest correlations with recidivism, but was not significantly greater than the SFS or Wisconsin RNA. Examined 27 effect sizes and found that the LSI-R works equally well for men and women.

Level of Service Inventory Revised (LSI-R); Salient Factor Score (SFS); Wisconsin RNA; Other (authors classified “other” as SFS clones that “contain about 5 to 10 items, almost all of which were static in nature”)

Which predictor domains and assessment instruments are the best predictors of adult offender recidivism? (Gendreau, Little, & Goggin, 1996)

Major Findings

Instruments Included

Research Question/Author(s)

Table 1.2  Overview of Systematic Reviews and Meta-Analyses Across Risk Assessment Tools

Level of Service Inventory Revised (LSI-R); Psychopathy Checklist Revised (PCL-R); Violence Risk Appraisal Guide (VRAG); Sex Offender Risk Appraisal Guide (SORAG); Static-99

Classification of Violent Risk (COVR); General Statistical Information on Recidivism (SIR); Historical Clinical Risk Management 20 (HCR-20); Historische, Klinische, Toekomstige-30 (HKT-30); Level of Service/Case Management Inventory (LS/CMI); Level of Service Inventory Revised (LSI-R); Minnesota Sex Offender Screening Tool Revised (MnSOST-R); Offender Group Reconviction Scale (OGRS); Psychopathy Checklist Revised (PCL-R); Psychopathy Checklist: Screening Version (PCL:SV); Psychopathy Checklist:Youth Version (PCL:YV); Rapid Risk Assessment for Sexual Offense Recidivism (RRASOR); Risk Matrix 2000 (RM2000); Spousal Assault Risk Assessment (SARA); Structured Assessment of Violent Risk in Youth (SAVRY); Sex Offender Risk Appraisal Guide (SORAG); Structured Outcome Assessment and Community Risk Monitoring (SORM); Static-99; Static-2002; Sexual Violence Risk 20 (SVR-20);Violence Risk Appraisal Guide (VRAG);Violence Risk Screening 10 (V-RISK-10); Violence Risk Scale (VRS);Violent Offender Risk Assessment Scale (VORAS); UK700

What is the predictive validity of tools commonly used to assess the risk of violence, sexual, and criminal behavior? (Fazel, Singh, Doll, & Grann, 2012)

What analytic approaches and performance indicators are most commonly used to measure predictive validity? How are analytic approaches and performance indicators defined and interpreted? (Singh, Desmarais, & Van Dorn, 2013)

Examined 251 studies and found that violence risk assessment tools performed best, and had higher positive predictive values than tools aimed at predicting sexual offending. Risk assessment instruments for general offending had lower diagnostic odds ratios and areas under the curve than the other two classes of instrument. Findings also indicate that the predictive accuracy of risk assessment tools varies depending on how they are used.  Found inconsistencies in AUC methodologies and benchmarks, most notably variation in benchmarks to determine AUCs that were small, medium, or large in magnitude. Findings suggest a need for standardization of predictive validity reporting to improve comparison across studies and instruments.

Faye S. Taxman with Amy Dezember

v­ alidity of the instrument (see Austin, 2006 for commentary on this; Durso, Caudy & Taxman, 2013). In fact, in most instruments, the static risk factors are stable in their predictive capacity whereas the dynamic factors vary considerable. This essentially means third generation tools that add the dynamic factors may not perform as well as second generation tools. Of course, the added dynamic risk items might be more useful for other goals, such as supervision planning or allocation of resources, but not for prediction of recidivism. The scoring of the RNA tool is another area where there is ongoing debate in the field. Some instruments have a total score that consists of static and dynamic risk factors and some instruments have separate scores. Baird (2009) contends that RNA tools that have a total score of static and dynamic risk overstate a person’s risk level since the dynamic factors do not improve the prediction. Instruments that have separate scores are more accurate in terms of the prediction of recidivism. Baird (2009) makes the argument that the static and dynamic risk should be scored separately. The measurement of items in the instrument is a final area where tool developers are not in agreement. Many of the dynamic factors include a variety of characteristics that are not specific, do not differentiate from past features compared to contemporary behaviors, and lack psychometrically sound scales to measure core factors. For example, in measuring dynamic risk factors, it is important to distinguish between whether the person is currently experiencing a problem behavior or has ever had past experience. That is, many of the instruments mix various constructs of how to measure problem behavior.The mixture includes: 1) lifetime or the individual has or had an event occur; 2) within the last 12 months or the individual has experienced the event within that period of time; and, 3) within the last 30 days or that the individual is more likely to be experiencing the event contemporaneously. The degree to which an item is “dynamic” is affected by whether it is measured as a lifetime or other factor. Lifetime events will overestimate a dynamic risk factor while other measures will focus on behaviors that are more interfering in a person’s current situation. Some of the dynamic risk factors (such as education, employment, and family/marital factors) are not necessarily predictive of recidivism at the bivariate or even in multivariate models, while other types of substance use are reported to be related to prediction (such as use of heroin and cocaine, frequent use and being an addict) (Taxman, 2014). More work is needed in defining the measurement facets of the dynamic risk factors, and how they are related to various outcomes. This is to prevent the over-representation of problem areas, which may serve to over classify individuals in higher risk categories. Are the statistical methods used in creating the tools appropriate and is the tool ­validated? As previously discussed, there is a science around risk prediction that involves ­methodologies and measurement of key variables. The methods for developing and validating the tool are critically important. Generally, there is a need to construct a tool with a different sample than one uses in the validation. This allows for the statistical models to be constructed and then tested on another data set, which ensures the robustness of the models. There are statistical applications to ensure that the models are predictive of the desired outcomes as well as meet the tests for construct validity and sensitivity analyses. All of these steps are critically important in deve­loping an instrument and ensuring that the instrument functions (performs) at a level that will be acceptable to improve decision-making in a justice setting. After all, the goal is to have RNA tools that perform better than clinical judgment (first generation) or chance. The tools need to be accurate, particularly with attention to avoiding Type I errors that under classify individuals or Type II errors (which are namely false positives) that over classify individuals in terms of risk levels. Statistical tools exist to measure the accuracy of the models as well as their ability to discriminate risk levels. Favored tools include the Receiver Under the Curve (ROC) and the Area Under the Curve (AUC); there are guidelines for what is acceptable level in the field. While some instruments report the ROC or AUC statistics, some do not (Singh, Desmarais, & Van Dorn, 2013). And, some RNA tools have acceptable levels while others perform under the desired standard. 10

The Value and Importance of RNA

Needless to say, it is critical to ensure that the tools that are used are meeting acceptable level of performance in order to ensure that the tools themselves are adding value to the decision-­ making. While researchers may accept a moderate or less performance for a tool, for the practice of justice this should not be accepted or tolerated since it diminishes the value of the RNA tool. Transparency in the methods for developing and validating the RNA tool is critically important to have overall confidence in the tools.Transparency is difficult because of the nature of some tools (proprietary versus public domain). But the confidence in the RNA is built on understanding that the proper steps were taken to validate the tool, and that the tool performs well in the intended setting. It is in these arenas where more information is needed to fully appreciate how well the RNA tool can improve decision-making.

Practice: Implementation Challenges with RNA Tools The “art” associated with RNA is the adoption and implementation of the tool and how the tensions regarding the value and utility of using RNA tools are handled. With a proliferation of RNA tools in correctional settings, research studies are beginning to document that in many cases “the emperor has no clothes”. That is, in both juvenile and adult probation settings, researchers frequently appear to be documenting that correctional staff/supervision officers regularly administer the RNA tool during intake, but the risk and need assessment information is seldom integrated into key decisions regarding resource allocation, case management, or supervision plans (Miller & Maloney, 2013; Viglione, Rudes, & Taxman, 2015; Bonta, Rugge, Scott, Bourgon, & Yessine, 2008; Flores, Russell, Latessa, & Travis, 2005; Luong & Wormith, 2011). This means that officers are taking the time to administer the instrument, probably to meet agency requirements, but are not using the information in routine practices. If RNA tools are meant to advance the goals of predicting recidivism and helping organizations to address recidivism related factors, then the failure to use the information gathered from RNA is a major limitation and challenge to the science. Implementation is a new area of science where attention is given to the nature of the intervention (RNA), inner factors, outer factors, characteristics of actors/individuals, and processes used in the organization. The Consolidated Framework for Implementation Research (CFIR) (­Damschrodier, Aron, Kirsch, Alexander, & Lowery, 2009) furnishes a conceptual approach to review RNA in practice settings. Taking an implementation science approach can improve the understanding of the linkage between the RNA tool (referred to as intervention characteristics in the CFIR) and the implementation of its use. Instead of defining the issue as “resistance”, more attention is given to the setting, environment and culture, users, external support and processes that act as barriers or facilitators of the use of the RNA tool. Of course, the concerns about implementation are worthy of considering, given that the failure to use the RNA tool results limits the benefits from the structured decision-making.

Nature of the RNA Tools The characteristics of the RNA tool—both in terms of the generation and the length of the tools, as well as the science behind the tool and other key characteristics—might affect the use in the field. The fifth generation tools (machine-learning) are based on the premise that having a risk score provided by information maintained in administrative databases will be easier for staff than completing a structured interview and scoring the results. In the prior section, we already identified a number of instrument specific information that can positively or negatively affect confidence in the RNA tool such as the ability to predict the desired outcome, the validity of the items in the tools, and the statistical methods to use to ensure predictive accuracy.We implied that there are a host of policies and procedures that an operational agency might need to have in 11

Faye S. Taxman with Amy Dezember

place to direct officers or line staff on how to use the RNA information in their daily practices, and that it is apparent that the utilization of the RNA tool depends on how the agencies intend for the information to be used. Problems of implementation can stem from a lack of agency ­policies, but it is equally likely that the complexity of the RNA tools interferes with the utilization of the information tools (Andrews & Bonta, 2010; Taxman & Belenko, 2012). Complexity of the tools is associated with the number of items on the tool, the different scoring methods, or the interpretation of the material. Another issue that can arise is that the officers do not agree with risk classification from the RNA tool, and therefore have little confidence in the tool results. Practitioners often do not incorporate information gained from RNAs into supervision and case management decisions because they do not trust the psychometric properties of the instrument (Krysik & LeCroy, 2002; Schwalbe,   iglione, Rudes & Taxman, 2015) or they prefer to use their own subjective judgment 2004; V instead of assessment tools (Andrews & Bonta, 2010; Gottfredson & Moriarty, 2006; Hilton & Simmons, 2001). Practitioners look at the discrepancy between the tool’s results and their knowledge of the person, and tend to doubt the accuracy of the RNA tool for “high stake” cases such as sex offenders or drunk drivers (Viglione, 2015). With the rise in the use of trailor instruments (i.e. instruments that are added to a general RNA to address special populations), there is an added burden to the overworked actor in a justice agency. Often agencies will select an instrument and then decide that there is a need for customization, such as tools specifically for sex offenders, to address gender responsiveness, to evaluate substance abuse, or to assess domestic violence. The added burden of these instruments is both in the time needed to use them and also integrating information from the general RNA plus the additional instruments. This may mean that an individual actor needs to reconcile any conflict results from these different tools (i.e. the person could be low risk on a sex offender instrument and higher risk on general recidivism), which adds to the complexity of using the tools. And, it means that the current items in the RNA tools are insufficient for decision-making. While many agencies emphasize and use validated versions of later generations of tools, there are still tensions about which tools are preferred and actually used. A few studies attempt to identify what type of tools have the highest degree of predictive validity—but the results vary and frequently depend on the how the RNA tool is used (Fazel, Singh, Doll, & Grann, 2012; ­Hanson & Morton-Bourgon, 2009). Because there is no clear consensus on which tool is preferred, the local socio-political environment affects which tool is selected and the varying practices of how to integrate second generation tools that only focus on risk factors. Static risk tools are simple to use, quick to complete, and generally feasible to use, yet they fail to assist practitioners to intervene and reduce offender risk (Viglione, Rudes, & Taxman, 2015). In fact, many agencies struggle between having a simpler tool or a comprehensive package of tools; the decision often centers on how intake occurs and the time that the agency is willing to allocate to assessment and diagnosis. Implementation issues hinge on the staff ’s knowledge on how to administer, score, and use the results in case planning and management, which often is lacking. In a survey of probation officers, researchers found that there is a wide variance in the levels of compliance with assessment tools and categorized the use patterns into noncompletion, careless tool completion, tool manipulation, and nonadherence to tool recommendations (Miller & Maloney, 2013). Similarly, in an ethnographic study of two probation officers, researchers found that while all POs routinely conduct the assessment, only one (out of 69) officer used the tool in a way that directly aligns with agency policy and training (Viglione, Rudes, & Taxman, 2015). Essentially, staff/officers lack sufficient knowledge to routinely use the information in practice. This is complicated by a lack of treatment services to refer clients to, which means that the officers do not view the value of translating risk and need information into opportunities for offenders to deal with criminogenic needs given the dearth of services (Taxman, Perdoni, & Caudy, 2013). 12

The Value and Importance of RNA

Inner Setting In implementation science, inner factors refer to the characteristics of the agency, such as n ­ etworks or communications among layers of staff, culture and climate, rationale for change, compatibility with existing practices, goals and organizational incentives and rewards. In other words, the inner factors refer to measurable characteristics of the agency regarding the support for the RNA tool. The implementation challenges are often deeply embedded in organizational issues that require changes in the culture, policies, or workflow process to improve the implementation effectiveness of RNA tools. While improved training for officers will help advance comfort with the tools, it does not address the challenges when the workflow does not support the tools. One major cultural issue is that probation and parole are historically compliance-driven probation agencies that emphasize rule obedience and meeting conditions of supervision. Under a compliance-driven model, officers are often constricted to monitoring the conditions of supervision instead of developing requirements based on the results from the RNA information tool. Another cultural issue has to do with the degree of compliance; while some agencies have graduated sanction policies, they tend to be undermined by line staff (Rudes, 2012; Turner, et al 2012; Steiner, 2011). ­Community corrections administrators should consider whether to use the standardized instruments to identify offenders for different ­levels of supervisory control or to refer offenders to treatment based on their needs (Viglione, Rudes, Taxman, 2015). In the case of community corrections, there are often many complexities and challenges associated with using RNA tools in practice, and the third and fourth generation tools can achieve the greatest results when the tools are used in various ways to facilitate case management and treatment planning. For correctional agencies, there is a need to clarify the intended purpose of using the RNA tool. In prison and jail settings, offender risk assessment is largely concerned with two major issues: security and programming. But these may be conflicting goals if administrators want to increase access to programming for higher risk offenders that may be a higher risk for security. Or, it could be that the needed programs for higher risk (with higher need) clients that are not available. Since RNA tools are promoted as assisting prisoners understand their likelihood of engaging in offending behavior and identifying criminogenic needs related to recidivism that can be addressed with programming (Makarios & Latessa, 2013), it is important that these are goals of the agency. Scenarios in which there is not a clear alignment between the goals for using the RNA tool and the purposes of a specific tool are likely to lead to improper use or implementation of the tool results in practice. It is important for the jail/prison administrations to help staff understand the value of the RNA tools given the goals and programming available in the institution, and to learn how the use of risk assessments for this purpose can lead to reduced institutional misconduct (French & Gendreau, 2006). Many agencies do not engage staff in the organizational change process as they embark on implementing a RNA tool in practice. Involving staff can facilitate discussions about work processes and procedures that are needed to foster greater support and utilization of the RNA tool. There are a myriad of quality improvement processes available that agencies can use to advance implementation (see Taxman & Belenko, 2012, Chapter 4).These processes foster staff involvement to accommodate changes in the environment to ready it for the use of the tool. In total, implementation is a challenge. Part of the challenge lies in preparing staff to use the tools but also having the corresponding policies and workflow to accommodate risk and need assessment information in practice. This is the art associated with these science-based tools.

Outer Setting Most justice agencies are part of a system that includes other processes and procedures such as the judiciary, prosecutors, defenders, prisons, jails, and community corrections. Each can provide external support and incentives for the use of RNA tools. The nature of the justice system in different 13

Faye S. Taxman with Amy Dezember

jurisdictions influences each other; yet, we typically do not have sufficient information about the system in a given jurisdiction and the support from one agency to another. This is typically an understudied area of research. As more justice organizations adopt RNA tools—and in some cases use different RNA tools—there is a greater need to understand the context of the system that affects utilization. For example, in one jurisdiction, the probation agency uses the COMPAS RNA tool and the prosecutor’s office is planning on using the STRONG-R. In assessing an individual, these two agencies may have different goals or results when using the RNA tool, which could result in conflicting risk levels that will have to be reconciled. The use of different RNA tools on the same individuals for different purposes in the justice system has not been examined but it has implications for the validity of the tools especially in the individual who is being assessed by various agencies that look at different features. Research is needed in how the use of various tools affects confidence in the results, how they are used, how it impacts justice and fairness in the system (based on different decisions at different points in the system), and how varied results are explained to the person.

Characteristics of Individuals Related to the inner setting, the CFIR model explores how the characteristics of staff using the RNA tool(s) affect the use of the tools. This includes the knowledge and belief in the RNA and the individual’s skill sets and other personal attributes that might affect the use of the RNA tool. As previously mentioned, Miller and Maloney (2013) examined the characteristics of the supervision officers by different characteristics of compliance with the RNA tools. They found that few demographic characteristics differentiated among the officers that were classified as substantive, bureaucratic, and cynical compliers (using latent class methods) except for the race/ethnicity of the individual officer. This is one of the few studies that examines the characteristics of officers on utilization patterns for the RNA. However, the study did find that bureaucratic compliers were less likely to have confidence in the RNA tools overall including the specific one used by their agency. Additionally, the less frequently the supervisor monitored how the tool results were going to be used, the more likely the officers were bureaucratic compliers. But if officers were trained on the RNA tool within the prior year, they were less likely to be bureaucratic compliers. Regarding cynical compliers, these officers were less likely to have confidence in RNA tools and have a perception that the agency is not enthusiastic for the tool. Administrative procedures that require approval of supervisors serve to reduce cynicism about complying with the tool.The opinions and perspectives, as well as experiences of the intended users, are relevant to the utilization of complying with the RNA tools. More studies of this nature are needed to understand the organizational factors that can support the use of the tools—this would guide others in their quest to advance implementation.

Processes Another area that is understudied but pertinent to implementation is the process for adoption and implementation of RNA in justice agencies. This includes the planning, engagement of staff in the implementation, development of opinion leaders and champions within the organization to support the use of the RNA tools, use of implementation leaders, and execution of the implementation plans. The process of implementation is important to achieve staff and agency confidence in the RNA tools, improve perception of the value and compatibility of the RNA tools with the agency mission, and advance an understanding of the feasibility of the RNA tool. Yet, the area of organizational change processes in justice systems has not been thoroughly studied. Many justice agencies are now using internal and external coaches to ease and accelerate implementation. But the questions are: what is the role of the coach, what are some of the functions that they perform, and how can one develop these coaches so that they are champions of the change processes? In a randomized study examining three types of coaching-related supports for the use of RNA in 14

The Value and Importance of RNA

juvenile justice settings—coaching to expand knowledge about the tools, coaching to build a social network to support the tools, or management directives regarding the importance of using the tools—juvenile justice staff were more likely to use RNA tools in case planning when the coach used a social network framework to develop the perception that the tools were consistent with agency goals (Taxman, Henderson,Young & Farrell, 2012). This understudied area is ripe to assist with advancing “substantive compliers” with RNA.

The Articles in this Volume This book examines the art and science related to risk and need assessment (RNA) tools for justice agencies. The gamut of topics covered include the history of risk and need assessment, ­methodological issues in creating and validating RNA tools, using assessments to improve correctional performance, and critical issues regarding the practice of RNA in justice organizations. In this collection, we explore the science of RNA as well as the art of implementation. This book contains 19 chapters that will provide a foundation of these topics and analyze different issues raised in this introduction. Bill Burrell (Chapter 2) begins a discussion of the history of risk and needs assessment and its use in probation and parole, and highlights the gap between theory and practice. This chapter reviews the barriers to implementation of RNA in probation and parole agencies and presents strategies that have been proven effective in successful implementation. In Chapter 3, Jim Byrne and Amy Dezember explore issues of implementation and use of RNA tools in prison settings and provide the perspective of correctional practitioners about the current use and future direction of assessment and classification tools. As noted earlier in the science behind RNA, methodological issues are critically important. ­Bobbie Brame (Chapter 4) explores the concepts behind static risk with attention to the measurement of risk as well as recidivism. He reminds us that there is much work to be done to further understand risk, particularly if we want to incorporate concepts of desistance or ensure that the risk assessment tools are developmentally appropriate. The need for gender or race neutral instruments, as often discussed, is understudied and needs more clarity to use in prediction efforts. Much of the discussion in this book is about the “new kid on the block” methods or the fifth generation of RNA tools (machine learning). Jim Hess and Susan Turner (Chapter 5) discuss some of the methodolo­gical issues with a comparison between fifth and third generation tools.This chapter compares the different approaches and then tests the value of adding geographical location to the prediction. KiDeuk Kim and Grant Duwe (Chapter 6) also provide an overview of various statistical methods to create RNA tools—the Burgess method, classical regression, and machine-learning algorithms—and then compare the prediction performance of these different methods. The authors provide a case study on the prediction of sexual offending among juvenile offenders and discuss the benefits and drawbacks of data-driven modeling for risk assessments. Similarly, Ira Schwartz, Peter York, Mark Greenwald, Ana Ramos-­ Hernandez, and Lisa Feeley explore the use of machine learning techniques in juvenile justice settings (­Chapter 7), where they examine how different approaches can minimize racial bias in assessment tools. Lastly,Tim Brennan (Chapter 8) discusses the complex predictive methods and machine learning by presenting the contradictions in the literature about these techniques. He provides some guiding principles to consider in using different statistical and machine learning techniques. The next section is devoted to issues regarding dynamic risk factors and responsivity for populations in RNA practice. Glenn Walters (Chapter 9) outlines how the criminal lifestyle and cognitive thinking should be better integrated into the theory associated with risk and need assessment strategies, and demonstrates the need for greater measurement of these constructs. Emily Salisbury, Breanna Boppre, and Bridget Kelly (Chapter 10) discuss the issues related to gender responsiveness in RNA instruments. While their study supports the need for further work on gender issues, they acknowledge the challenges in developing some of these measures. Karl Hanson and Guy Bourgon 15

Faye S. Taxman with Amy Dezember

(Chapter 11) discuss sex offender instruments and the importance of measurement of both gene­ral recidivism and sex offending. In this chapter, they also advocate for a common language as a means of communicating risk level and identify that one of the underdeveloped characteristics of the RNA technology is the categorization of risk. Katherine Ginsburg-Kempany and Kimberly Kaiser (­Chapter 12) identify that many RNA studies fail to include measures of procedural justice and legitimacy, which might advance an appreciation for some untapped dynamic factors. The authors make the argument that individuals who have low confidence in the justice system may present different dynamic risks than others. Garima Siwach and Shawn Bushway (Chapter 13) discuss the use of risk and need assessment tools for employment decisions, a new area of research and application of RNA. Finally, Brittany Via, Amy Dezember, and Faye Taxman (Chapter 14) explore the inconsistencies among the measures of dynamic risk factors and the need for better consensus on these domains. They raise the question as to what the different domains mean since there are few similarities among five instruments.This chapter reviews five instruments to demonstrate an unmet need in the science field to improve our knowledge of the relationship between dynamic risk factors and recidivism. The next section of this book discusses implementation of the RNA and its use in practice. ­Zackary Hamilton, Elizabeth Thompson Tollefsbol, Michael Campagna, and Jacqueline van Wormer (­Chapter  15) highlight the need for customizing assessment tools to enhance implementation and utilization and provide some basic principles for this customization. Joel Miller and Sara Trocchio (Chapter 16) reconceptualize the implementation processes, including the concept of the “street-level bureaucrat”, in relation to understanding how risk assessment tools are used in practice. Danielle Rudes, Jill Viglione, and Kimberly Meyer (Chapter 17) examine how practitioners use risk assessment tools by providing findings from adult and juvenile justice agencies on the probation officers’ perceptions of risk assessments and observations of how POs understand and navigate needs when making critical decisions about supervision level, treatment planning, service referrals, and ongoing supervision decisions. Lastly, the book ends with a few chapters addressing special issues regarding the conceptualization of risk and needs assessments. Kelly Hannah-Moffat (Chapter 18) raises the issues related to how RNA could be just another version of systemic bias or institutional biases. She questions some of the issues related to how much RNA can be racially, gender, or age neutral given the proxy to other variables. Mary Rogan (Chapter 19) addresses the human rights issues associated with classifying individuals as high risk. The author argues that based on international human rights treaties, risk and need assessments can help address how to properly classify high-risk offenders and provide access to rehabilitation to address criminogenic needs and encourage desistance.

Future Directions While RNA has progressed through various generations to improve decision-making and provide greater predictive validity and guidance for case planning, there is still the need for further research to be conducted to improve the effectiveness of RNA in practice. This handbook aims to discuss some of the areas where research is needed, while also proposing future direction for research. Collectively the articles present areas where further research, ranging from design of the RNA tool to utilization in the field, is needed. These seminal papers will help better understand some of these issues while continuing to push for advancements in how RNA tools can be used to assess recidivism, improve risk assessment methods, advance offender supervision, and better allocate resources. Table 1.3 summarizes some major research questions. Let us conclude this introduction with these future directions.

RNA Tool Design and Characteristics (Intervention) In general, a number of unresolved questions exist that need exploration. One key issue is the predictive accuracy of the tool, and how best to score the tool to maintain that accuracy. Included is whether different methods for creating tools are superior to any other method including machine 16

The Value and Importance of RNA

learning. And then there is a need to explore how best to score the tools including whether to have a single score or a separate score for risk and needs.These issues relate to the predictive accuracy of the RNA tool as well as the goal for the tool. A number of contributors to this volume identify other variables that should be included in the RNA tool, or at least explore the feasibility of adding such variables such as location, residence, race, age, and gender. Race, age, and gender are three variables that are typically not included in most tools but these variables are related to recidivism. And, sometimes there are proxies for these variables. In the past, these variables have not been used under the guise of being “neutral” on demographics. But more and more attention is now being given to transparency with a push towards having different tools based on race, gender or age. All issues for pending research.

Inner Setting Utilization of the RNA tools by practitioners is still a very understudied area. A few studies ­document that the tools are not as valued by the users, and that the systems have not fully integrated the RNA tools into practices. Essentially, an understudied area of research has to do with a better understanding of the workflow process including where and how to integrate RNA into case planning, supervision meetings, compliance issues, and other common parts of the process. Some of the unresolved issues have to do with staff confidence in the RNA tools, but others pertain to conflicting goals in an organization. And, how the organization demonstrates commitment to staff using the tools to guide decisions has yet to be resolved.

Outer Setting Typically outer setting issues pertain to the support for the use of RNA tools (and other interventions) by the justice organization.The external support is viewed as leverage to further support the use of the tool internal to the organization. But, as the number and type of RNA tools proliferates, more attention will be needed on how the practices of one organization affect another, especially when there are different tools that have different goals and objectives. The complementary and/or conflicting nature of the various tools may become more of an issue that will need further exploration in the future.

Characteristics of Individuals Few studies have explored the characteristics of the users of the RNA tools and different facets that affect utilization. As shown in the Miller & Maloney (2013) article, much can be gleaned from how to improve utilization if we better understand and appreciate the perspectives of those using the instruments. Multilevel studies that combine inner setting and outer setting issues are important since they will examine the degree to which organizational factors (i.e. goals, priorities, policies, etc.) and individual characteristics of the users affect how the RNA tool is used. Such explorations can foster more information about utilization that might be beneficial to advancing implementation.

Process Implementation of RNA is still a black box of how an organization should employ RNA innovations. In the general literature, there is a focus on change teams, quality improvement processes, ­champions, opinion leaders, and other diffusion techniques. In justice settings, there are few studies that give insight into which process works best given the array of inner and outer setting issues. Involving staff is considered an important factor but the processes that use staff in these change environments are not well understood. More research is needed overall in how to affect change in the short and long term. 17

Faye S. Taxman with Amy Dezember Table 1.3  Future Research to Advance Risk and Needs Assessments Intervention Characteristics

Does using general pool assessment items improve accuracy and workflow? Where and when to use non-traditional techniques for scoring RNA tools? How to incorporate the concept of criminal lifestyle into RNA tools? How to incorporate procedural justice in RNA tools? How to improve RNA tools through customization? How to decide what outcome measure to use when assessing recidivism risk? How to make decisions about cut points or developing standardized risk categories? What role can “big data” play in improving machine learning techniques? Inner Factors How to increase the workflow process to improve the implementation effectiveness of RNA tools? How to increase the degree of compliance by line staff ? How to identify organizational commitment to implementation and measure success of adoption? Outer Setting How to rectify the use of different RNA tools on the same individuals for different purposes in the justice system? How can others make use of the RNA tools? How do judges and other criminal justice practitioners use and weigh risk information? Characteristics of Individuals How do the individual characteristics or demographics of agents administering RNAs impact the degree of compliance? How can training be improved to increase compliance and RNA adherence? Process How to engage staff in organization change process? What role can internal and external coaches play in improving implementation? How to develop leaders and champions to garner support for RNA tools? How to create and execute implementation plans? How to bridge the gap between research and practitioners to increase agent confidence in assessment tools?

Conclusion Risk and need assessment tools are powerful innovations in justice settings where science and practice complement each other. Many have argued that we have come a long way in the deve­ lopment of the tools, and that they are beneficial to the justice system to improve decision-making. This is not doubted, but as shown by the collection of articles in this Handbook, we have a lot more to learn about developing, refining, implementing and using instruments in justice settings to advance decision-making. The proliferation of instruments and utilization is valuable but the scholars contributing to this volume clearly indicate a need for more research to advance science and practice. We need to recognize what we know about RNA instruments, and where gaps exist in our knowledge that affect both the science and practice. This Handbook outlines an agenda to begin to answer many of these questions.

Note 1 Center for Advancing Correctional Excellence (ACE!) George Mason University

References Andrews, D. A. & Bonta, J. (2010). The psychology of criminal conduct. Routledge. Andrews, D. A. & Bonta, J. (2010). Rehabilitating criminal justice policy and practice. Psychology, Public Policy, and Law, 16(1), 39–55.

18

The Value and Importance of RNA Andrews, D. A., Bonta, J., & Wormith, J. S. (2006). The recent past and near future of risk and/or need assessment. Crime and Delinquency, 52(1), 7–27. Austin, J. (2006). How much risk can we take-The misuse of risk assessment in corrections. Fed. Probation, 70, 58. Baird, C. (2009).A question of evidence:A critique of risk assessment models used in the justice ­system. ­Madison, WI: National Council on Crime and Delinquency. Bonta, J., Rugge, T., Scott, T. L., Bourgon, G., & Yessine, A. K. (2008). Exploring the black box of community supervision. Journal of Offender Rehabilitation, 47(3), 248–270. Boothby, J. L. & Clements, C. B. (2000). A national survey of correctional psychologists. Criminal Justice and Behavior, 27(6), 716–732. Brennan, T. (1987). Classification for control in jails and prisons. Crime and Justice, 323–366. Brennan, T., Dieterich, W., & Ehret, B. (2012). Evaluating the predictive validity of the COMPAS risk and needs assessment system. Criminal Justice and Behavior, 36(1), 21–40. DOI: 10.1177/0093854808326545. Campbell, M. A., French, S., & Gendreau, P. (2009). The prediction of violence in adult offenders a meta-­ analytic comparison of instruments and methods of assessment. Criminal Justice and Behavior, 36(6), 567–590. Casey, P.M., Warren, R.K., Elek, J.K. (2012). Using Offender Risk and Needs Assessment Information at Sentencing: Guidance for Courts from a National Working Group. Williamsburg,VA: National Center for State Courts. Doi: http://www.ncsc.org/~/media/Microsites/Files/CSI/RNA%20Guide%20Final.ashx. Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science, 4(50). Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34(7), 571. Desmarais, S. L. & Singh, J.P. (2013). An executive summary drawn from Assessing Recidivism Risk: A Review of Validation Studies. New York: Council of State Governments Justice Center. Fazel, S., Singh, J. P., Doll, H., & Grann, M. (2012). Use of risk assessment instruments to predict violence and antisocial behaviour in 73 samples involving 24 827 people: systematic review and meta-analysis. BMJ, Jul 24:345. doi: 10.1136/bmj.e4692. Review. PMID: 22833604. Flores, A. W., Russell, A. L., Latessa, E. J., & Travis, T. F. (2005). Evidence of Professionalism or Quackery: ­Measuring Practitioner Awareness of Risk/Need Factors and Effective Treatment Strategies. Federal ­Probation, 69(2). French, S. A. & Gendreau, P. (2006). Reducing Prison Misconducts What Works!. Criminal Justice and Beha­ vior, 33(2), 185–218. Gendreau, P., Little, T., & Goggin, C. (1996). A meta-analysis of the predictors of adult offender recidivism: what works!. Criminology, 34(4), 575–608. Gottfredson, S. D. & Moriarty, L. J. (2006). Statistical risk assessment: Old problems and new a­ pplications. Crime & Delinquency, 52(1), 178–200. Grove,W. M. & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures:The clinical–statistical controversy. Psychology, Public Policy, and Law, 2(2), 293–323. Hanson, R. K. & Morton-Bourgon, K. E. (2009). The accuracy of recidivism risk assessments for sexual offenders: a meta-analysis of 118 prediction studies. Psychological assessment, 21(1), 1–21. Hastie, R. & Dawes, R. M. (2010). Rational Decision in an uncertainty world: The psychology of judgment and decision making. Sage Publication. Hilton, N. Z. & Simmons, J. L. (2001). The influence of actuarial risk assessment in clinical judgments and tribunal decisions about mentally disordered offenders in maximum security. Law and Human Behavior, 25(4), 393–400. Hoffman, P. B., & Beck, J. L. (1997). “The Origin of the Federal Criminal History Score.” Federal Sentencing Reporter 9(4), 192–197. Krysik, J. & LeCroy, C. W. (2002). The empirical validation of an instrument to predict risk of recidivism among juvenile offenders. Research on Social Work Practice, 12(1), 71–81. Luong, D. & Wormith, J. S. (2011). Applying risk/need assessment to probation practice and its impact on the recidivism of young offenders. Criminal Justice and Behavior, 38(12), 1177–1199. Makarios, M. & Latessa, E. J. (2013). Developing a Risk and Needs Assessment Instrument for Prison Inmates: The Issue of Outcome. Criminal Justice and Behavior, 40, 1449–1471. Mamalian, C. A. (2011). State of the science of pretrial risk assessment. Gaithersburg, MD: Pretrial Justice Institute. Miller, J. & Maloney, C. (2013). Practitioner Compliance With Risk/Needs Assessment Tools A Theoretical and Empirical Assessment. Criminal Justice and Behavior, 40(7), 716–736. O’Leary,V. & Clear,T.R. (1984). Directions for Community Corrections in the late 1990s. Washington, DC: National Institute of Corrections. Petersilia, J. & Turner, S. (1993). Intensive probation and parole. Crime and Justice, 281–335.

19

Faye S. Taxman with Amy Dezember Rudes, D. S. (2012). Framing organizational reform: Misalignments and disputes among parole and union middle managers. Law & Policy, 34(1), 1–31. Schwalbe, C. (2004). Re-visioning risk assessment for human service decision making.  Children and Youth Services Review, 26(6), 561–576. Singh, J. P., Desmarais, S. L., & Van Dorn, R. A. (2013). Measurement of predictive validity in violence risk assessment studies: A second-order systematic review. Behavioral Sciences & the Law, 31(1), 55–73. Smith, P., Cullen, F.T., & Latessa, E. J. (2009). Can 14,737 women be wrong? A meta-analysis of the LSI-R and recidivism for female offenders.Criminology & Public Policy, 8(1), 183–208. Steiner, B., Travis, L. F., & Makarios, M. D. (2011). Understanding parole officers’ responses to sanctioning reform. Crime & Delinquency, 57(2), 222–246. Taxman, F.S. (2002). Supervision–Exploring the dimensions of effectiveness. Federal Probation, 66(2), 14–27. Taxman, F.S. (2014). Second Generation of RNR: The Importance of Systemic Responsivity in Expanding Core Principles of Responsivity. Federal Probation. http://www.uscourts.gov/uscourts/FederalCourts/ PPS/Fedprob/2014-09/rnr.html. Taxman, F.S. & Belenko, S. (2012). Implementation of Evidence Based Community Corrections and Addiction Treatment. New York, NY: Springer. Taxman, F.S., Henderson, C.,Young, D.W., & Farrell, J. (2014). The impact of training interventions on organizational readiness to support innovations in juvenile justice offices. Administration of Mental Health Policy and Mental Health Services Research. 41(2): 177–188. DOI: 10.1007/s10488-012-0445-5. Taxman, F.S., Perdoni, M., & Caudy, M. (2013). The plight of providing appropriate substance abuse treatment services to offenders: Modeling the gaps in service delivery. Victims & Offenders, 8(1), 70–93. The Pew Center on the States. (2011, September). Risk/Needs Assessment 101: Science Reveals New Tools to Manage Offenders (Issue Brief). Retrieved from http://www.pewtrusts.org/~/media/legacy/uploadedfiles/ pcs_assets/2011/pewriskassessmentbriefpdf.pdf. Turner, S., Braithwaite, H., Kearney, L., Murphy, A., & Haerle, D. (2012). Evaluation of the California parole violation decision-making instrument (PVDMI). Journal of Crime and Justice, 35(2), 269–295. Viglione, J. (2015). Bridging the Research/Practice Gap: Street-Level Decision Making and Historical Influences Related to Use of Evidence-Based Practices in Adult Probation (Doctoral dissertation). George Mason University, Fairfax,VA. Viglione, J., Rudes, D. S., & Taxman, F. S. (2015). Misalignment in Supervision Implementing Risk/Needs Assessment Instruments in Probation. Criminal Justice and Behavior, 42(3), 263–285. doi: 0093854814548447. Wormith, J. S. (2001). Assessing offender assessment: Contributing to effective correctional treatment. The ICCA Journal, 12–23.

20

PART I

NA History of RNA

This page intentionally left blank

2 RISK AND NEEDS ASSESSMENT IN PrOBATION AND PArOLE The Persistent Gap Between Promise and Practice William D. Burrell

Introduction Risk and need assessment (RNA) is a common element of contemporary probation and parole practice in the US. There is widespread agreement that RNA is an important component of offender supervision case management systems, and risk and need assessment instruments and policies are routine components of probation and parole supervision. The term “risk/needs” is a regular element of discourse in agencies of all sizes and jurisdictions (­Gerlinger and Turner 2015). Despite the seemingly all but unanimous endorsement from the field, there is substantial evidence that the endorsement of RNA may be more rhetorical than operational. Adoption is not universal, and implementation is uneven and often inadequate. Alexander and her colleagues write: “Although many probation departments, both state and federal, have claimed to use risk assessments in supervision for decades, in most cases the reality is that they administer the risk assessments but fail to use them to adjust supervision commensurate with risk.” (Alexander,Whitley and Bersch, 2014, p. 2). Resources are wasted by supervising offenders too much or too little, opportunities for risk reduction are being squandered, community safety is compromised, and expensive jail and prison resources are overused. This results in much of the potential of RNA being lost due to improper or inadequate implementation.

The Importance of RNA The importance and centrality of RNA to corrections is captured well by Wormith and his colleagues in this opening paragraph to their recent article. Offender risk assessment has become a mainstay activity of correctional agencies worldwide, both in the community and in custodial institutions. It is used for a variety of ­decisions made by criminal justice officials in the classification, management, and supervision of offenders. Clinicians and other practitioners may also use risk assessment in planning a course of treatment and rehabilitation programming for the offender. Increasingly, correctional administrators are utilizing risk assessment to control the size of the prison population, and to rein in the cost of operating local, state, and national correctional agencies. The extent to which these diverse kinds of uses will serve such functions depends 23

William D. Burrell

on the quality of the assessment and the subsequent decisions and placements that are based on them. (Wormith, Hogg and Guzzo 2015, p. 461) Wormith and colleagues are not alone in this sentiment. Harris states that “(a)ctuarial risk prediction is now a fundamental precept of evidence-based practices” (Harris 2006, p. 8). Mulvey adds “assessment of risk is a central feature of juvenile justice, both for policy and daily practice” (Mulvey 2005, p. 461). Holsinger and colleagues call risk and need assessment central to a case classification system (Holsinger, Lurigio & Latessa, 2001). Latessa and Lovins find RNA to be the foundation of effective correctional practices (Latessa and Lovins, 2010). Finally, Flores and colleagues conclude that “actuarial risk assessment is a veritable cornerstone for the provision of correctional services” (Flores, Lowenkamp, Holsinger & Latessa, 2006, p. 524). There can be little doubt that RNA is a critical element of evidence-based supervision of both adult and juvenile offenders in the community. There is a broad consensus on this point in the academic, research, and policy arenas. As Young and colleagues note, “The call to apply researchbased findings has been made perhaps most persistently in the area of assessment.” (Young, Moline, Farrell & Bierie, 2006, p. 136). The importance of using decision tools to make the best, most appropriate decisions is borne out by the life-changing consequences that attach to decisions in the juvenile and criminal justice systems. Community corrections agencies, courts and paroling authorities should endeavor to ensure that their decision-making systems are as sound, fair and just as possible.

History Risk and needs assessment has a more than 40 year history in probation and parole supervision. The roots of risk assessment for the purposes of predicting offender behavior go even farther back, almost a century. The first efforts to use actuarial methods to predict parolee success as part of parole release decision-making originated in Illinois in the 1920s (Harcourt, 2007). Those efforts and others using historical offender data and actuarial methods in parole release decisions continued through the first half of the twentieth century (Harcourt, 2007). The work on statistical models of prediction picked up speed in the 1960s with the development of the Base Expectancy Score (BES) instrument for parole release decisions in California (Gottfredson and Gottfredson, 1993). In New York City, the Vera Institute of Justice developed an instrument for pretrial release in the Manhattan Bail Project in 1964 (Katzive, 1968). The 1967 President’s Commission on Law Enforcement and the Administration of Justice’s Task Force on Juvenile Delinquency and Youth Crime featured a paper by Don Gottfredson entitled “Assessment and Prediction Methods in Crime and Delinquency” (Gottfredson, 1967). Scientific methods were beginning to make a mark in the assessment of adult and juvenile offenders. The 1970s saw the development of the Salient Factor Score (SFS) risk assessment by the United States Parole Commission as part of the work to develop parole guidelines (Hoffman and Beck, 1974). Work on parole guidelines in several states further raised the awareness of prediction instruments in the field (Gottfredson, Cosgrove, Wilkins, Wallerstein and Rauh, 1978). The decision to release an inmate to the community on parole is directly linked to the risk of reoffending. Parole statutes routinely require the paroling authority to assess the risk to the community when considering an inmate for release on parole (Rhine 2012; Rhine et al., 1991). As risk assessment expanded in parole release decision-making, awareness of the technology spread to probation supervision. This was an area that was largely uninvolved with scientific assessment methods. In 1976, the Government Accounting Office (GAO) issued a report on state and county probation calling them “systems in crisis” (Government Accounting Office, 1976). The GAO noted that probation agencies need to focus attention and services on those who need the 24

Risk and Needs Assessment in Probation and Parole

most assistance and supervision. They noted that better decision-making tools, such as risk assessment instruments, could be helpful in this process. But probation was devoid of risk assessment instruments and instead relied on personal experience and professional judgment (GAO 1976, Bohnstedt and Geiser, 1979). As caseloads grew, managers sought ways to more effectively utilize their limited staff and optimize the allocation of resources to provide greater service and surveillance to those who pose greater risk (Bohnstedt and Geiser, 1979). Probation and parole agencies began to see the value of risk assessments. By 1979, a survey of probation and parole agencies found that 30% of agencies (105/350) surveyed reported using classification instruments (Bohnstedt and Geiser, 1979). Almost half (44%) of those agencies adopted an instrument developed elsewhere. The most commonly adopted instruments were the Wisconsin RNA, the California parole BES, the federal parole SFS and the Missouri Client Analysis Scale (Bohnstedt and Geiser, 1979). The watershed project for RNA in probation and parole was the Case Classification and Staff Deployment (CC/SD) project of the Wisconsin Bureau of Community Corrections. Begun in 1974 as an effort to rationalize and justify staffing requests to the state legislature, the CC/SD project produced the first fully integrated case classification and case management project in the US. The model included an actuarial risk assessment, a needs assessment, risk and need reassessments, the Client Management Classification (CMC) system (a case management and treatment model) and weighted workload standards to determine staffing needs (Baird, Heinz and Bemus, 1979; ­Bohnstedt and Geiser 1979). With the publication of a two-year follow-up report on the project (Baird, Heinz, and Bemus, 1979), word began to spread throughout the field through conference workshops and professional networks. When the Wisconsin system was adopted as the core the National Institute of Corrections’ (NIC) Model Probation and Parole Classification and Case Management System (Model System) in 1980, the field quickly embraced “risk/needs” (National Institute of Corrections, 1980). Through the Model System project, agencies in 43 states received training and technical assistance to implement the model, including risk and needs assessment (Burke, 1990). While some agencies chose to utilize other risk and needs instruments, the majority chose Wisconsin’s tools (Burke, 1990). As these US agencies implemented the Model System and worked with it over time, the next innovation in RNA technology was developing in Canada. The Level of Supervision Inventory (later renamed the Level of Service Inventory) or LSI incorporated dynamic risk factors that identified targets for correctional interventions as well as a means for empirically assessing offender change (Andrews and Bonta, 2010). This concept was presented in the seminal 1990 articles by Andrews and colleagues that introduced the risk/needs/responsivity principles, or R/N/R (Andrews, Bonta and Hoge, 1990; Andrews, Zinger, et al., 1990). This developed in parallel with the “what works” movement, a return to correctional treatment that emerged in response to the “nothing works” mantra that grew out of Martinson’s infamous article on the effectiveness of correctional treatment (Martinson, 1975). The late 1990s and early 2000s saw the development of enhanced RNA models that explicitly link the assessment to case planning and supervision strategies. Prominent among these are the ­Correctional Offender Management Profiling for Alternative Sanctions or COMPAS (­Northpointe Institute for Public Management, 2015) and the Level of Service/Case Management Inventory or LS/CMI (MHS, 2015).

The Generations of Risk and Need Assessment As risk assessment has developed over the past almost 100 years, new developments in statistical practices, assessment technologies and correctional treatment research have resulted in several major shifts in the practice of risk assessment. James Bonta first classified these developments into ­generations of risk assessment (Bonta, 1996). 25

William D. Burrell

The first generation (1G) of risk assessment is clinical/professional judgment. With this type of assessment, the practitioner (a probation officer in this case) interviews the offender and reviews available reports and files. Based on that information, the officer applies expertise gained through experience and practice to make a judgment about the level of risk. This assessment is unstructured and idiosyncratic, opaque and largely unreviewable. It is subject to bias, and to the application of preconceived and often incorrect notions.This model requires experience and practice, which puts the new practitioner at a distinct disadvantage and increases the possibility of error due to lack of experience. The second generation (2G) represents a significant improvement through the introduction of statistical methods. These instruments are based on actuarial analysis of previous experience with offenders. The analysis identifies those variables that are statistically related to the outcome of interest, in this case, reoffending. The variables with the greatest explanatory power are assembled into an instrument. In contrast to 1G assessments, the 2G are structured, consistent, transparent and reviewable. They are based on empirical analysis, are unbiased and reliable. Research has shown the 2G instruments to be more accurate than 1G approaches across a variety of fields (Hilton, Harris, & Rice, 2006). The third generation (3G) represents another leap forward. Variables in the 2G instruments were static, such as characteristics of offenders and their experiences in the past that are fixed and will never change. For one time decisions such as release on parole, assessment based on fixed factors was acceptable. As correctional research and practice evolved and began to emphasize programming and treatment, a RNA needed to be able to capture changes in the offender’s risk and need profile. The 3G instruments incorporated dynamic risk factors, which are variables related to reoffending which were changeable through the application of therapeutic techniques and services. These dynamic factors, also referred to as criminogenic factors, not only enabled probation and parole officers (PPOs)1 to periodically reassess risk and needs as part of managing supervision, they also provided targets for treatment, services and interventions. The fourth generation (4G) instruments go beyond assessment of dynamic risk and need factors and link the assessment to a case management plan. This helps to ensure that risk factors identified in the assessment are addressed in the supervision. This provides a more systematic approach. The 4G instruments also incorporate a broader range of factors important to correctional treatment, such as offender strengths and responsivity factors (Bonta and Andrews, 2007).

The Promise of Structured Decision-Making Writing in 1987, Gottfredson and Tonry made this prescient prediction: “Both the literature and practical applications of science-based prediction and classification will continue to expand as institutions evolve to become more rational, efficient and just.” (Gottfredson and Tonry, 1987, p. vii). Indeed, the emergence of the third and fourth generation of RNAs and the development of myriad specialized assessment for sub-populations of offenders validate their statement. Writing almost 20 years later, Byrne and Pattavina note that RNA is undergoing major change, with new instruments, redefined roles and responsibilities for staff and redesigned case management systems based on these emerging technologies (Byrne and Pattavina, 2006). The research literature on and the practical applications of RNA have continued to expand. The present challenge is to drive the implementation deeper to the point where RNA is a core element of the organization’s culture and practice. Reviewing the literature on PPOs and structured decision-making, one cannot help but conclude that most staff have little idea why actuarial models are so widely implemented in community corrections (Taxman and Belenko, 2012). Beyond any cynical desire to “scientificate” probation and parole (Schneider et al., 1996), the power behind the argument for actuarial instruments can be summarized easily in language that is familiar to any consumer—they are better, faster and cheaper at assessing risk. 26

Risk and Needs Assessment in Probation and Parole

Actuarial assessment models are better in that they are more accurate than human judgment, they are unbiased, they are transparent, and they provide consistency, uniformity and standardization of practice—all things that are enormously difficult to achieve when relying solely on human judgment. These assessment instruments are faster—the underlying empirical research has already identified the most powerful variables in predicting recidivism, so the PPO doesn’t need to waste valuable time wading through mountains of information, some of which is irrelevant to the question of risk. Even the third and fourth generation RNAs, while admittedly longer in terms of sheer number of variables, save time in the identification of areas for intervention and in formulation of case plans and supervision strategies. Again, PPOs can focus quickly on what is important and what works, because the developmental research has targeted the key information and eliminated the need to go searching for what may be important. Because they are faster to complete than a quality clinical interview and investigation, actuarial instruments save time and resources. Less time spent by the PPO doing an assessment can result in more time available to spend with clients providing supervision. This relates to a common complaint among PPOs—they don’t have enough time to work with their caseload (Latessa and Lovins, 2010). Gottfredson and Moriarty (2006b) note that risk assessment instruments “increase the reliability and prognostic validity of decisions” (p. 16). If the instruments were only equal to human judgment in accuracy, they are better because they are faster, saving time and resources. In addition to saving time and resources, Silver and Miller note that RNAs are a tool for managing institutional resources, facilitating the efficient management of resources by basing decisions on statistical relationships instead of intuition and conjecture (Silver and Miller, 2002). In her study of adult parole in California, Lynch noted that risk assessments were part of a process that also provided standardization of supervision practices, established expectations for levels of supervision (contact standards), generated documentation of parole agent activities and provided an overall system of accountability (Lynch, 1998). These are all valuable elements of a sound management system for case supervision. While many who look at actuarial risk and need assessment may only see through the narrow lens of risk scores, it is clear that full implementation of a comprehensive assessment system provides a great deal more value to the probation and parole agency than an empirically grounded decision about who to supervise and how. (See Appendix 1 for the elements of a comprehensive RNA system.)

Actuarial Versus Clinical Judgment It is worth spending a bit of time exploring the distinction between actuarial risk and need assessments and clinical judgment, as this issue forms the heart of many of the arguments about and resistance to structured decision-making instruments. The issue is multifaceted and practitioners should know the full picture, as that may reduce if not eliminate much of their resistance to actuarial RNA. This information should include how actuarial instruments are developed, limitations of human judgment and of actuarial instruments, methods for use and information on effectiveness. Simply stated, clinical judgment is unstructured investigation and evaluation of risk of reoffense described above. In addition to the limitations noted, the clinical model is also constrained by the physical limitations of the human brain for making decisions. The brain is capable of considering only a limited number of variables at a time and as new information is added, some information drops out (Miller, 1956). Further, recall is limited in capacity and accuracy. A PPO may have interviewed thousands of clients, but how much information can be recalled in useful format? ­Additionally, is the information recalled relevant to the case at hand? Or is it memorable to the PPO for some reason other than relevance to risk? Silver and Miller state, “… actuarial risk 27

William D. Burrell

assessment tools are capable of combining more information than could the typical human expert.” (Silver & Miller, 2002, p. 139). Research on this approach is clear. As Harris notes: “No strong empirical case can be made for risk assessment based on unstructured clinical judgment.” (Harris 2006 p. 10). In an examination of federal probation officers and their risk assessment capabilities, Oleson and his colleagues (2011) found that probation officers were more accurate and consistent in assessing risk when they used the RNA than when they used their own professional judgment. The officers were given a case scenario to review and assess for risk, using their own judgment. They were subsequently trained on the RNA and asked to reassess the case scenario using the RNA. The second round of assessment was more accurate (more officers determined the correct score) and more consistent (91% of the officers agreed on the level of risk) (Oleson, VanBenschoten, Robinson, and Lowenkamp, 2011). Additionally, Perrault and colleagues found that PPOs tend to overestimate risk, often considerably, when using their own professional judgment. PPOs rely on gut instincts and personal experiences.They often over emphasize prior record and substance abuse, and are prone to conflate risk and offense seriousness (Perrault, Paiva-Salisbury, and Vincent, 2012). In contrast, actuarial assessment instruments are based on empirical research that systematically examines an issue (in this case recidivism) by analyzing large data sets (hundreds and often thousands of cases) with known outcomes to determine which characteristics of the subjects explain the variance in outcomes. In other words, what are the characteristics of offenders who recidivate compared with those who don’t recidivate? The variables (characteristics) that explain the greatest amount of variance in outcomes are assembled into an assessment instrument. The scoring system typically represents the explanatory power of the variable (how much of the difference in outcomes its presence or absence can account for). Variables with greater explanatory power get greater weights in the scoring system. Items such as race with obvious ethical implications are excluded. A critical point in this discussion is that actuarial instruments do not produce specific predictions about individual offenders. Rather they produce probability statements for groups of offenders with certain characteristics related to reoffending. An actuarial analysis will allow cases to be sorted into groups, such as low, moderate and high risk. Each group will have an aggregate failure rate, for example a 40% chance of reoffending for the high-risk group. The actuarial technology can’t determine which specific member of that group will reoffend, just that the probability of reoffending for the group is 40%. Actuarial models shift the focus from individual judgments for specific offenders based on professional expertise and experience to aggregate judgments based on statistical analysis. Knowledge about large numbers of offenders, their characteristics and behavior, organized in a useful and targeted manner, is much more helpful to the decision-maker (Mulvey and Iselin, 2008). Stephen Gottfredson summarized the research in this area. “In virtually every decision-making situation for which the issue has been studied, it has been found that statistically developed predictions devices outperform human judgments.” (S. Gottfredson, 1987, p. 36). A comprehensive review by Dawes, Faust and Meehl (1989) confirms that conclusion that actuarial methods outperform clinical judgments across a variety of settings (psychology, mental health and general medicine) and tasks (personality assessment, medical prognosis, intellectual testing). Given that actuarial models outperform clinical judgment, one might feel justified in recommending full reliance on actuarial RAI and the elimination of professional judgment altogether. Indeed, many practitioners have mistakenly concluded that their agency leaders are advocating such an approach. As Harris has described it, there is an erroneous belief that use of a RAI requires PPOs to abandon carefully cultivated professional judgment to determining offender risk (Harris, 2006). Balucci (2012) provides two examples of misperceptions about RNA and practitioners. First, that the RNA prevents the PPO from including certain information about the client. And second, that the RNA eliminates the possibility that a PPO will let clinical, professional or personal 28

Risk and Needs Assessment in Probation and Parole

opinion shape an assessment. Neither perception is correct and there are two concerns here. First, if leaders of community corrections agencies advocate an approach that prohibits the exercise of professional judgment by staff, they are seriously mistaken. Actuarial instruments have their limitations and should not be relied upon in isolation of professional judgment. Second, if practitioners are getting that message in error, then there are significant problems with the messages delivered to staff about implementation of the RNA and the training provided to staff. Both points will be addressed later in this chapter.

Professional Judgment Schneider and colleagues note that critics of actuarial RNA “defend discretionary decisions on the grounds that there is not a system of rules or scientific aids that can anticipate all of the possible contingencies and variances in human behavior that will actually be encountered.” (Schneider et al., 1996, p. 111). That is correct, and well-designed assessment systems address this concern by encouraging thorough review and assessment beyond just the variables on the RNA to ensure that a full picture of the client and all relevant factors has been developed. Actuarial instruments contain a limited number of variables by design and by the results of the statistical analysis. In order to be useful to the practitioner, a limit on the size of instruments has to be established. The analysis that identifies and weights the items on an RNA identifies the most powerful variables. At some point, the amount of additional explanatory power added by the next variable in the sequence is too little to justify its inclusion.That does not mean that everything that might be predictive of risk for every offender is included. A list of all factors would be so large as to be unmanageable to use in assessing risk, not to mention all but impossible to compile. The practical implications of this limitation of actuarial instruments point directly to the use of professional judgment. In conjunction with an actuarial instrument, professional judgment can identify individual factors which may be very relevant for the offender being assessed, even if not for the entire population. PPOs can identify events in an offender’s life or characteristics not included in the RNA, which may not be generally predictive for the offender population as a whole but are for this offender at this point in time (Gottfredson and Moriarty, 2006b). Perrault and colleagues note that professional judgment can account for low frequency but acute situations, like episodic bouts of serious illness, and for idiosyncratic factors that are specific to this offender’s situation, such as loss of a job or divorce (Perrault, Paiva-Salisbury, and Vincent, 2012). Incorporating such information into the assessment may result in an override of the instrument’s recommendation. This is a critical function that enables the assessor to tailor the aggregate analysis to the individual offender. Research into actuarial decision-making has shown that combining professional judgment with actuarial analysis produces even better, more accurate decisions (Gottfredson and Moriarty, 2006b). Darlington summarized this issue succinctly: “This research does not suggest that human judgment is generally unnecessary; rather it indicates that the most accurate predictions generally result from a predictive system in which human judgment and statistical analysis are combined according to prescribed rules.” (Darlington, n.d). This is not a new finding by any means. The fourth principle in Andrews and Bonta’s model (after risk, need and responsivity) is professional discretion, for just this purpose, and this concept was first articulated some 25 years ago (Andrews, Bonta and Hoge, 1990; Andrews, Zinger, et al., 1990). In their recent guidebook, Vincent and her colleagues also recognize the professional discretion principle (Vincent et al., 2012). Outside the field of probation and parole, the discussion has been for some time how to use both modes of assessment in combination (Gottfredson and Moriarty, 2006b). Exercise of professional discretion through an override is something that must be managed carefully to avoid abuse and manipulation. Miller and Maloney note that overrides are a gray area in 29

William D. Burrell

practice, sometimes appropriate, sometimes not. They must be carefully monitored through supervisory review and approval of each instance where an override is requested. According to Byrne and Pattavina (2006), overrides should be limited to 10% of assessments, while others call for a limit on override of between 5 to 10% of all assessments (Vincent et al., 2012). Many agencies have implemented policy-driven overrides, where certain classes of offenses or offenders are presumptively classified, regardless of risk score. Examples include overriding to the highest supervision level for sex offenders or domestic violence offenders due to the high stakes of new offending and the lack of, or lack of confidence in, specialized risk assessment for these cases. In some resource constrained agencies, misdemeanor offenders or diversion cases are presumptively assigned to administrative supervision. Actuarial instruments provide useful and powerful tools to PPOs, tools that can improve ­decision-making and save time. But they are not infallible or omniscient. Used well by the decision-­maker in conjunction with professional experience and expertise, they enhance practice significantly.

Structured Professional Judgment Instruments There is a third option between purely clinical judgment and the structure of an ­actuarial ­instrument. Structured professional judgment (SPJ) instruments combine elements of both approaches and come with benefits and costs. According to Harris (2006), SPJ assessments are formal instruments, developed empirically and designed and validated on the target population. They only contain variables that have been demonstrated to correlate with the criterion outcome (recidivism). In this manner, they ensure that only relevant information goes into the assessment. The scoring of the SPJ provides more flexibility to the PPO, which results in a more positive view of the instrument and its utility. These instruments take longer to administer and require additional skill on the part of the PPO, so there are costs related to their use compared with actuarial instruments. There is the need for additional training and monitoring to ensure that the flexibility provided in administration does not allow bias or other unwanted factors to enter the assessment process. According to Perrault, “actuarial and SPJ risk assessment tools are comparable in terms of their predictive accuracy for reoffending.” (Perrault 2012, p. 489). Agency decision-makers should carefully assess the costs and benefits of any RNA, including the SPJ. A SPJ instrument will take more time to train staff and may take more time to administer, but if PPOs like it better because it is less mechanical and allows them to make greater use of their expertise, it may be worth the additional cost because there will be better assessments due to increased utilization and potentially better outcomes.

Extent of Use of RNA In the three decades and more that RNA has been a part of probation and parole, how have things changed? Is RNA still a common component of practice? Has it been strengthened or has it receded and been replaced by something else? In a field as decentralized and fragmented as probation and parole in the US, it is challenging to determine and then describe policy and practices in anything but the broadest of terms. Detailed information is hard to collect and even harder to interpret, given the persistently local nature of American government and the diversity of practices in those locales. Despite these challenges, numerous surveys have attempted to document the extent of use of RNA in adult and juvenile probation and parole in the US. Latessa (2004) reported that 75% of probation and parole agencies use RNA and that 83% of the respondents to the survey said that RNA is absolutely important or very important. An NIC survey found 97% of community corrections agencies using RNA (Brumbaugh and Steffey, 2005). Similarly, the Urban Institute studied utilization of RNA in adult parole and found that 82% of parole supervising agencies responding 30

Risk and Needs Assessment in Probation and Parole

(30 of 36) reported using RNA always or most of the time (Jannetta, Elderbroom, Solomon, Cahill, Parthasarathy, and Burrell, 2009). In the juvenile system, Schwalbe reported a significant increase in use of RNA, from 33% prior to 1990 to 85% in 2008 (Schwalbe, 2008). Guy and colleagues reported that in 1989, 47% of juvenile courts and probation used formal RNA and then in 1992, another survey reported that most of the states used structured decision-making that assessed risk factors (Guy, Nelson, Fusco-Morin, and Vincent, 2014). However, not all surveys report such high rates of utilization. Perdoni and colleagues reported that 34% of community corrections agencies use RNA (Perdoni, Taxman, and Fletcher, 2008). These data, while inconsistent in some ways, do appear to confirm that RNA is widely used in the field. It is fair to assume that a good deal of the increased utilization has been driven by the growing knowledge in the field of the principles of evidence-based supervision generally and more specifically the risk/need/responsivity (R/N/R) model of Andrews and Bonta (2010). The larger question is what do these data mean? How should a response to a survey question that claims utilization of RNA be interpreted? This is an area where knowledge is incomplete. Schneider and colleagues note, In spite of the enormous increase in reliance on scientific decision aids, very little is known about how these instruments have been implemented, how they are used, and how they are viewed by those whose discretion is being curtailed. (Schneider, Ervin, and Snyder-Joy, 1996, p. 110) This dearth of knowledge about RNA use at the line level was also noted by Miller and Maloney (2013) and Hass and DeTardo-Bora (2009). Utilization of RNA varies widely in practice. At one extreme, it is a simple ‘check the boxes’ scoring of a second generation risk assessment (static factors), using whatever information is readily accessible to determine supervision level and then supervising as usual, meeting contact standards and enforcing conditions of supervision. At the other extreme of utilization would be full use of a fourth generation (dynamic and static factors) risk and needs tool. Full use would include personal interview(s) with the client and perhaps others, review of related reports and materials, scoring of the instrument, discussing the results with the client and collaboratively creating a case plan addressing criminogenic factors, conducting supervision targeting risk reduction, and regularly reassessing risk and needs and updating the case plan and supervision strategy. The differences between these two approaches should be clear, as should their potential for reducing risk and facilitating reintegration in the community. Unfortunately, either approach could legitimately be scored on a survey as “using RNA in supervision”. Full utilization is the only way that the full potential of RNA can be realized. Doing the minimum (checking the boxes) is little more than going through the motions. No positive impact of any significance should be expected from that approach. Time spent just ‘checking the boxes’ is largely wasted—something that probation and parole can ill afford.

Shift in Correctional Philosophy and Practice The emergence of actuarial RNA coincided with a seismic shift in the philosophy of corrections in the US. Beginning in the late 1970s and continuing for better than 20 years, the core assumptions and values of sentencing and corrections shifted from an indeterminate sentencing system with a core commitment to offender rehabilitation to a determinate sentencing system where rehabilitation was devalued, if not dismissed outright and replaced with clearly retributive goals. The core value of corrections became punishment, and the primary techniques for probation and parole were surveillance and enforcement. At the same time, the correctional systems in the US, both institutional and community-based, began to grow exponentially. 31

William D. Burrell

The increase in the corrections population translated into a new approach for probation and parole. Risk control and risk management became paramount concerns. Protecting community safety was the primary goal, and offender rehabilitation faded from view. Feeley and Simon labeled this shift in philosophy as the “new penology” (Feeley and Simon, 1992). Actuarial risk assessments provided the means to identify risk levels and thus target higher risk offenders for surveillance, enforcement and control. There was little tolerance for non-compliance with conditions of supervision and technical violations became the tool for controlling risk. Enhanced enforcement efforts such as intensive supervision and electronic monitoring upped the ante on risk control. The American system of law focuses on the individual. The actions of the individual that violated the law, culpability for those actions and consequences are all individually based. Corrections strategies were also individually based. The shift to aggregate analysis of risk and development and implementation of policies based on the results also represents a significant shift. Offenders who were classified as high risk based on the profiles of others were subject to enhanced control and management efforts. While straightforward and justifiable, given rising crime rates and the political tenor of the times, this shift has had lasting and fundamental implications for correctional philosophy and practice. This revolution has gained additional traction with the latest iteration of the actuarial approach to justice—the emergence of risk-informed sentencing (Barry-Jester, ­Casselman, and Goldstein, 2015). Insurance companies are perhaps the most prominent users of the actuarial model to determine individual level consequences from aggregate data. Reichman (1986) describes an insurance-based system of risk control in corrections. The emphasis in this approach is to anticipate and minimize crime. Anticipating is assessing risk and minimizing is exerting control through surveillance and enforcement. In this model, little or no attention is paid to the causes of crime or to efforts to ameliorate the causes. Risk management is a corporate terminology, driven by aggregate data to identify those situations that may result in a loss. Reichman states: “A risk management approach to crime does not offer any promises to eliminate crime by seeking out and correcting its underlying causes by rehabilitating offenders.” (Reichman, 1986, p. 164). Silver and Miller (2002) describe this actuarial justice environment as one where the goal is to minimize the harm of deviance through risk control. “The abandonment of efforts to identify the origins of deviance (and to correct them) seems to be an inevitable byproduct of the use of actuarial tools.” (Silver and Miller, 2002, p. 149). Those offenders who violate the conditions are revoked and incarcerated, with no effort to understand why they were out of compliance. In this era of risk control, high revocation rates for technical violations were evidence of effectiveness at identifying and removing risk prone offenders from the community (Silver and Miller, 2002). This phenomenon also appeared in juvenile probation, although to a lesser degree. Mulvey and Iselin note that “… professionals are now more inclined to identify variables that raise or lower the probability of violence and the methods for managing them.” (Mulvey and Iselin, 2008, p. 45). What has declined as risk has become more prominent is the interest in addressing treatment needs of juveniles on probation. It is not difficult to see how this “new penology” conflicts with the rehabilitative ideal that characterized corrections, particularly community corrections, prior to 1975. In her study of California parole agents, Lynch describes how the shift to actuarial methods met with serious resistance from the agents. Agency leaders preferred control-based interventions over treatment or service interventions, representing a shift in the philosophy of parole (Lynch, 1998).

The Reality on the Ground While little may be known about practice in the field, a series of studies and surveys provides a look inside a number of adult and juvenile agencies as well as national level data from surveys of PPOs in the US. Unfortunately, the information that has been gathered paints a rather dismal picture 32

Risk and Needs Assessment in Probation and Parole

of the utilization of RNA in both adult and juvenile probation and parole. Addressing juvenile justice, Mulvey and Iselin state that “juvenile justice professionals make limited use of these tools” (structured decision-making instruments such as RNA). Rather, probation officers make decisions based in intuition and experience. “(I)t is the exception rather than the rule to consider … empirically verified data.” (Mulvey and Iselin, 2014, pp. 38–39). In their research into adult parole and reentry, Haas and DeTardo-Borin found that the majority of staff (correctional case managers, counselors and parole officers) do not use the results of the RNA when developing reentry plans for adult offenders. Parole officers used the LSI-R to determine the risk score and classification, and establish the supervision level for the parolee—‘checking the boxes’ and no more (Haas and DeTardo-Borin, 2009). Viglione and colleagues agree that the use of RNA at the officer level is understudied. They found marginal use of the RNA, describing the process as ‘score the instrument, put it in the file and then supervise as the PO wants to’ (Viglione, Rudes, and Taxman, 2015).They found that POs prefer their own subjective judgment, “despite the vast amount of research questioning the validity of unstructured decision-making.” (Viglione, Rudes, and Taxman, 2015, pp. 265–6). PPOs emphasize risk and deemphasize or ignore needs altogether. In this study, the authors concluded that practitioners do not understand the purpose and background of the RNA, do not see the value in the RNA nor believe its use is appropriate. Not only are the PPOs in this study not using the RNA to its full potential, their practices are not in alignment with agency policy (Viglione et al., 2015). Despite being trained on the RNA, the PPOs revert to risk-focused supervision and do not address criminogenic needs.They are reluctant to address criminogenic factors because they do not believe that they can be changed or lack of skills in behavior change. In their study of probation supervision in Canada, Bonta and colleagues found that the PPOs failed to address criminogenic needs, instead focusing on enforcement of conditions. There was a gap between the RNA and the case management activities—the assessment was done, but the results did not make it into the supervision plan or PPO activities (Bonta, Rugge, Scott, Bourgon and Yessine, 2008). Lack of full or appropriate use of RNA in community supervision can take a number of forms (Miller and Maloney, 2013). They include: 1 Non-use—PPOs just don’t complete the instrument. 2 Careless administration—Staff use whatever information is available to score the RNA, not fully investigating, interviewing the client (and others) or verifying the information. 3 Instrument manipulation—Scores on individual variables or the instrument as a whole are manipulated to achieve the desired score and to affect the workload implications of the score. 4 Non-adherence to recommendations—PPOs disregard or only partially comply with guidance from the RNA. In their survey of PPOs in the US, Miller and Maloney found that 64% of the respondents said that they “sometimes or always” ignored problems (needs) to reduce the score on the RNA. A slightly smaller portion of respondents (58%) said they “sometimes or always” scored the RNA incorrectly to manage their workload, reducing scores to lower their workload (Miller and Maloney, 2013). In their study of PPOs in a midwest state, Schneider et al. found that PPOs were manipulating the scoring of the RNA by inappropriately overriding the RNA score, ignoring need areas and by ignoring the RNA score and supervising the client at the level that the PPO found appropriate (Schneider et al., 1996). Drawing on the work of Robinson and McNeill (2008), Miller and Maloney further discuss PO compliance with RNA policies. There are “formal compliers” who fill out the form but do little more.These staff can be further divided into “bureaucratic compliers” who fulfill their bureaucratic obligation but preserve their autonomy and use their individual clinical judgment to override the 33

William D. Burrell

instrument. “Cynical compliers” are careless in their completion of the RNA and tend to manipulate the scoring to get the result they want.These staff appear to consciously subvert the purpose of the instrument. In contrast to the formal compliers are the “substantive compliers” who complete the form and then go on to use the results in the development of a case plan and implementation of the supervision strategy (Miller and Maloney, 2013). This substantive compliance would be the preferred mode for use of RNAs. Some of the non-compliance or misuse of RNA may stem from what Viglione et al. (2015) see as the legacy of risk in probation and parole. Many agencies adopted risk assessment in the 1980s, which was the “second generation era” of risk assessment. Second generation instruments (Bonta, 1996) use mainly, if not exclusively, static factors to assess risk, and they formed the basis of the risk control model. Dynamic risk factors (needs) are not included on second generation risk assessment instruments. Many community corrections staff, particularly managers and executives, grew up professionally in the risk control era and cling to that model. In many ways, they can be labeled the post-Martinson generation, entering the field after the rehabilitative ideal (Allen, 1981) had been rejected and the driving force for probation and parole supervision became risk control to protect community safety. Many agencies and their staff have not made the transition to the dynamic era of risk and need assessment, ushered in by Andrews and his colleagues with the introduction of dynamic risk factors and the risk/need/responsivity principles (Andrews, Bonta and Hoge, 1990; Andrews, Zinger, et al., 1990). As Young and his colleagues note, risk and need assessment technologies have developed and advanced, but many staff persist in their reliance on professional judgment and intuition (Young, Moline, Farrell, and Bierie, 2006). At best they are still using 2G risk instruments. A second challenge identified by Viglione et al. (2015) is PPO fear of liability. If an officer, using the RNA instrument, classifies an offender as low risk and the offender commits a new crime, the PPO fears being held liable for the adverse consequences of that decision. This represents a fundamental misunderstanding of liability law and government employee immunity statutes and liability protections. Scoring the RNA and following the recommendations as prescribed by agency policy is a requirement or “ministerial duty” and thus is not grounds for a finding of liability. Whatever claim is made for negligence and liability is transferred to the agency that developed and adopted the policy.Thus following agency policy and working in good faith is the best way to avoid liability (Lyons and Jermstad, 2013). This information about the manipulation of and carelessness in administration of RNA is troubling. At a minimum, agency policy is being ignored. Some low risk clients will be over-­ supervised, wasting resources. They may be referred to unneeded programs or services, which can be iatrogenic. Other higher risk clients may be under-supervised, posing a public safety risk. Treatment and service needs go unaddressed, prolonging involvement in the correctional system, increasing costs and allowing offending to continue. As Miller and Maloney (2013) note, the effectiveness of the RNA is heavily dependent on practitioner conformance to the assessment process. The promise of RNA requires proper completion of the instrument and acting appropriately on the results.

What Accounts for the Gap Between Research and Practice? As Gottfredson and Moriarty note: “Properly developed and implemented risk assessment instruments can improve criminal justice decisions, properly target and potentially save resources and potentially increase public safety. Much of this promise remains largely unfulfilled.” (Gottfredson and Moriarty, 2006a, p. 195). Young and colleagues agree, noting that PPOs continue to rely on their own judgment and experience despite the advances in RNA technology (Young et al., 2006). The urgent question that remains to be answered is why, in the face of all of this potential for significantly improving practice and results, does probation and parole remain so far behind? 34

Risk and Needs Assessment in Probation and Parole

The answer to this question has multiple critical components, including the strategic management of the agency, its organizational culture, staff perceptions, the level of buy-in and commitment of staff, resources and implementation issues.

Strategic Management Strategic management is the big picture component of agency management. How an agency views its mission or purpose, its vision and philosophy of supervision is critical to the fate of RNA. Is the mission one of risk control or risk reduction? The latter implies a commitment to identifying and addressing risk factors that drive reoffending. Such a mission will form a strong foundation for full use of RNA, not just ‘checking the boxes’. Lack of commitment at the agency leadership level to risk reduction as a mission is a critical shortcoming. The first strategy in the Urban Institute’s EBP model for parole supervision is “define (agency) success as recidivism reduction and measure performance.” (Solomon, Osborne,­Winterfield, Elderbroom, Burke, Stroker, Rhine, and Burrell, 2008, p. 8). The logic of the model is that once an organization fully commits to reducing recidivism—new criminal activity—as its mission, the commitment to implement EBP and RNA will follow. Maintaining a risk control model will not provide motivation to implement RNA and will in fact inhibit adoption and implementation. It is essential to create an environment that is supportive of RNA specifically and EBP more generally. Implementation of an innovation such as RNA requires attention to the vision of the agency. Vision is the future that the agency is committed to creating. Failure to establish a compelling strategic vision for any proposed change can be a fatal error. The implementation of RNA will disrupt the routines and practices of the probation and parole agency. People resist change and any successful implementation effort will have to be built on a compelling vision of the future that will motivate staff and stakeholders to engage and embrace the vision of the future. “People need something they can see, at least in their imagination. They need a picture of how the organization will look and they need to be able to imagine how it will feel to be a participant in it” (Bridges, 2003, p. 55). Agency leaders must be the champions of change. Implementing major organizational change requires agency leaders to make personal and professional commitments to success of the change. Half-hearted support by leadership will be detected by staff and communicate to them that this is not critically important and thus compliance is not mandatory. A key aspect of being a change champion is communication of the ‘what’ and the ‘how’ of the change. Staff need to know how their duties and responsibilities are going to change and why. People can handle a good deal of change if it is coherent and part of a larger whole that they understand (Bridges, 2003). Communication needs to be relentless, ongoing, varied in platform, honest, upbeat and engaging.

Organizational Culture An underappreciated aspect of life in a contemporary agency is its organizational culture, which can simply be defined as “the way we do things around here”. Culture cannot be seen or held or touched, yet it is a powerful force that shapes behaviors better than any policy or training program can. Schein identifies key elements of culture as shared assumptions, which are driven by staff beliefs and values (Schein, 2010). The attitudes, values and beliefs of probation and parole staff about RNA shape their behavior and their utilization of RNA. Reviewing the research discussed below, it seems that many staff can be characterized as anti-actuarial, pro-professional judgment when it comes to the assessment of offenders. Vincent, Guy, Gershenson, and McCabe (2012) conclude that PPOs undervalue RNA and as a result, under-utilize them in practice. Failure to recognize and address culture can quickly doom organizational change or reform efforts to abject failure. Young et al. describe staff in one agency with a jaded view of the agency 35

William D. Burrell

leadership and their attempts to impose change from the top down. The new administration was determined to reform operations even before they know what was going on. Staff had predictably (and justifiably) dubious views of the executives who did not really know the system they were determined to reform (Young et al., 2006). Lynch describes parole agents who viewed agency management as outsiders, not completely trustworthy or supportive of the field agents. As a result, the parole agents actively subverted directives from management (Lynch, 1998). The importance of staff views is affirmed by Peterson-Badali et al., “The probation officers’ knowledge, skills and beliefs about the RNR principles and practices have been recognized as potentially important determinants of whether the framework will be implemented as intended” (Peterson-Badali, ­Skilling, and Haqanee, 2013). Efforts to change the culture and make it more receptive to change can markedly increase the prospects of successful implementation. Failure to build a supportive culture is a common shortcoming of organizational change strategies. Many executives are unaware of the culture of their organization and how important that is to the success of change efforts. The power of organizational culture is reflected in the phrase “culture will eat strategy for lunch”. Culture is that powerful and thus is critical to success of implementation.

Perceptions of Professionalism and Expertise Much of the resistance to actuarial RNA appears to be rooted in the desire to maintain professional judgment and discretion in the assessment of offenders. Maupin (1993) states that RNA system is perceived as an imposition on the PPO’s authority to determine how to supervise an offender. Mulvey and Iselin broaden the focus beyond PPOs. “Actuarial methods also contradict the view of the professional, whether the judge, the probation officer or a social service provider, as having unique knowledge and skills gained from years of practice.” (Mulvey and Iselin, 2008, p. 46). Lynch notes that parole agents believed they didn’t need anything but their own skills and experience to assess risk, and they gave very little credence to the value of RNA and case plan (Lynch, 1998). Summarizing the comments of a RNA critic, Fitzgibbons puts succinctly and accurately the perspective of many PPOs: “Horsefield questions whether such systems add anything at all apart from spurious scientificity. Most probation officers with any experience, he says, know precisely who is and who is not likely to reoffend.” (Fitzgibbons, 2007, p. 90, citing Horsefield, 2003). Ferguson notes the loss of discretion of the PPO leads to the perception of erosion of professional judgment (Ferguson, 2002). Young and his colleagues note that juvenile PPOs resent the RNA as an attempt to limit their discretion (Young et al., 2006). It is interesting to note that this is not strictly an American phenomenon. Robinson, writing about the implementation of RNA in England and Wales, describes resistance to the actuarial approach from probation staff based on their “reluctance to forgo the traditional, individualized basis of probation practice” (Robinson, 2002). In other words, probation is a people business and statistical tools cannot replace the experience, expertise and judgment of the PPO. Robinson (2003) further explores this issue in the UK and parallels abound. She describes professional insecurity of probation staff that flows from what she calls “deskilling” and “de-professionalization” of staff through the erosion of professional discretion. Robinson relates this to the reduction in the degree of “indeterminacy” in probation practice. In the past, the job of the PPO was vested with a great deal “indeterminacy”, that is the details of assessment and supervision were left largely up to the discretion of the PPO. As practices become increasingly standardized, routinized and programmed, indeterminacy is reduced and the professional status of the PPO is undermined (Robinson, 2003). Probation and parole staff have great confidence in their decision-making abilities. Harris describes the “belief in the supremacy of professional judgment about the offender’s likelihood 36

Risk and Needs Assessment in Probation and Parole

of new criminal behavior is prevalent in community corrections.” (Harris, 2006, p. 8). Latessa and Lovins summarize the prevailing attitudes on RNA more succinctly: “I don’t have time (to do the RNA). I just need to talk to them.” (Latessa and Lovins, 2010, p. 215). In a survey of adult PPOs, 76% believed that they should have more discretion in selecting the level of supervision and 61% believed that their professional experience and knowledge is superior to what the RNA could offer (Schneider et al., 1996). PPOs are reluctant to permit the quantitative prediction systems to replace their professional judgment. They resent the RNA for trying to quantify human behavior. In the survey of PPOs, only 13% felt that the RNA could do a better job than the PPO (Schneider et al., 1996). From these surveys and research, it is clear that many PPOs don’t believe in the accuracy of actuarial models and thus find them wanting when compared with their own judgment and expertise. The value that the RNA brings to the decision-making and case planning process is not seen or appreciated by the PPOs (Viglione et al., 2015). There is an interesting paradox around the concern of “de-professionalization” of the role of the PPO through use of RNAs. Many of the credentialed “professionals” in the field of human services, such as psychologists, licensed clinical social workers, addiction treatment specialists and others rely extensively on assessment instruments to help with diagnosis and development of treatment plans for their clients. Review of one vendor’s most recent catalog reveals some 130 assessment instruments for adult and juvenile client assessment and evaluation (MHS Inc., 2015). If probation and parole staff wish to be considered more “professional” (a common lament among PPOS in the US), one would expect them to embrace assessment instruments. An alternative view of actuarial and other assessment instruments would be to see them as a supplement to PPO judgment, not a replacement. Haas and DeTardo-Bora suggest that RNA should be presented to PPOs as a supplement to their work in order to lessen their fears that they are being replaced by an instrument (Haaas and DeTarbo-Bora, 2009). In a recent article about automation replacing human employees in industry, Davenport and Kirby introduce the concept that automation is an “augmentation” to human judgment and reasoning. They suggest starting with what employees do now and then exploring how that work can be deepened rather than diminished by greater use of automation (Davenport and Kirby, 2015). The parallel to actuarial assessment for probation and parole is easy to see. RNA can make the assessment process deeper and more sophisticated. Davenport and Kirby note that for the augmentation approach to work, managers must be convinced that the combination of humans and computers is better than either working alone. This reinforces how important the professional override concept is to the success of actuarial assessment systems. Whatever approach is used, the attitudes and beliefs of PPOs concerning RNA must be surfaced, addressed and changed. This is a critical roadblock to full implementation of RNA.

The Challenge of Staff Commitment The structure and mode of operation for probation and parole make buy-in and commitment of staff to organizational mission and policies crucial. Commitment is the promise to faithfully carry out agency mission and policies to the best of one’s abilities. PPOs are vested with tremendous discretion in the execution of their duties. In this sense they are a classic example of Lipsky’s street level bureaucrat (Lipsky, 1980). Their role is critical because “… the decisions of the street-level bureaucrat, the routines they establish and the devices they invent to cope with the uncertainties and work pressures effectively become the public policies they carry out.” (Lipsky, 1980. p. xii). Working at the “retail level” of justice (Mulvey, 2005) or “at the coal face” as the British term it, the commitment and cooperation of the PPO is essential to successful implementation of RNA in specific and organizational policy in general (Maupin, 1993). 37

William D. Burrell

Lynch’s study of parole agents demonstrates this point. Organizational policy doesn’t get implemented without some reshaping by those charged with carrying out the policy. Field agents serve as policy implementers so it is essential that they buy into the policy (Lynch, 1998). Maupin (1993) observes the PPOs are physically separated from their supervisors for large parts of the day, if not consecutive days. This is especially true with agencies that emphasize fieldwork and those in rural areas where the routine home visit can take hours, if not all day. Even with a structured RNA in place, PPOs still exercise considerable discretion in the delivery of supervision and other services. Working in such an independent manner makes understanding of and commitment to carrying out agency policy all the more critical for probation and parole. Despite the seemingly obvious nature of the importance of the street-level bureaucrat, it is often ignored. Harris and Smith (1996) write of the need for organizations and executives to develop street-level commitment among staff, particularly line officers. They note that “(t)he lack of attention to those who will implement an innovation is sometimes shocking” (p. 210). Too many managers assume that policies will be implemented if they just issue a directive and tell staff what to do. As Mulvey and Iselin note:“Methods (like RNA) are rarely adopted enthusiastically by juvenile justice professionals.” (Mulvey and Iselin, 2008, p. 44). PPOs need to see the value added by RNA and commit to full implementation in their work. Demonstrating the value added by actuarial RNA is particularly important given the discussion of PPO attitudes towards actuarial methods discussed immediately above. Lipsky (1980) concludes that “(w)e are not prepared as a society to abandon decisions about people and discretionary interventions to machines and programmed formats.” (Lipsky, 1980, p. xv). The actions of the street-level bureaucrats “are the policies provided by government in important respects.” (Lipsky, 1980, p. xvi). Ensuring adequate knowledge of and commitment to organizational policies is a key responsibility of agency management.

Resource Issues During the adoption and institutionalization of RNA in community corrections, both probation and parole caseloads grew significantly. Adults under community supervision increased by 3.7 ­million or 356% between 1980 and 2008. Probation added 3.1 million persons, an increase of 354% and parole added 633,472 persons, a gain increase of 347% (Bureau of Justice Statistics, 2015). In juvenile probation, the growth was less substantial, but still significant. Court dispositions to formal probation supervision increased by 67% between 1985 and 2008 (Perrault et al., 2012). This caseload growth and the challenge of optimally allocating staff drove many agencies to examine and adopt risk assessment instruments (Bohnstedt and Geiser, 1979). While the caseloads were growing, new responsibilities were also added that increased demands on PPO time. Drug testing became almost universal, electronic monitoring was added, a range of new fees and financial penalties were created and had to be collected and monitored, community service was implemented and intermediate sanctions programs were developed. While some of these came with additional resources, most did not. In this environment, RNA was just another responsibility to be added to the growing list. Individual officer caseloads grew along with the list of new responsibilities and there was simply not enough time to perform all of the mandated activities on all cases. Something had to give, and often it was the RNA. In such an environment, PPOs will find devices to cope with work pressures (Lipsky, 1980) and those devices will most likely target the policies and practices to which the PPO is least committed. Failure to allocate resources to support agency mission and vision is a key issue. Maintaining existing structures and resource distribution is rarely adequate to support full implementation of RNA. Reorganization of functions and staffing, and shifting of resources to support RNA sends a powerful message to staff about the agency leadership’s commitment to change. 38

Risk and Needs Assessment in Probation and Parole

Poor Implementation The history of evidence-based practice in community corrections in the US is a checkered one. A limited number of jurisdictions have implemented complete, integrated systems, but far more have implemented only pieces of the EBP model and far too often, have done even that poorly. The issue of implementation looms large over the field. The body of knowledge about what do to in order to improve effectiveness is large, robust and growing. What is lacking is a commensurate level of knowledge about how to put these practices into operation. This is the challenge for now and into the foreseeable future (Burrell, 2012; Rhine, Mawhorr, & Parks, 2006). The foundation for successful implementation of EBP is adequate knowledge. Many executives and managers simply do not know enough about the specific details of EBP and RNA. Senior leaders must become conversant with this body of knowledge and be able speak knowledgeably and credibly about it to create understanding and commitment within and outside the organization. Another area with knowledge gaps is the change process. Too many agency executives either misunderstand or are uninformed about the process of organizational change. Ballucci notes that RNAs are resisted and subverted by PPOs at the line level (Ballucci, 2012). Implementation of new practices such as RNA must be assertively and proactively managed if they are to be fully implemented with fidelity. RNA systems are often adopted but rarely are they fully integrated into agency policy and practice, and into case management systems, whether automated or manual. Without such integration, RNA will be marginalized and staff will seek to and succeed at manipulating the system or ignoring it altogether. Policies and practices must be aligned with the mission. It is critical to ensure that the organization’s formal footprint—the policies and procedures—reflect and support RNA.The policies represent what is required of staff and they must be in alignment with and supportive of RNA. The instrument selection process is often flawed. As Young et al. note, in the rush to implement RNA, agencies take tools off the shelf without paying attention to critical implementation issues such as staff acceptance, transferability and training (Young et al., 2006). Agency executives must realize that RNA is not a panacea for all of the ills of the organization (Mulvey and Iselin, 2008). Care must be taken in selecting an RNA, as one size does not fit all (Latessa, 2003; Gottfredson and Moriarty, 2006a).VanVoorhis and Brown conducted a review of risk classification for NIC and found a number of implementation problems. They include poor fit between the RNA, agency and decisions, failure to validate imported instruments, unreasonable expectations (RNA expected to compensate for inadequate resources), failure to link RNA to case plans, supervision strategies and treatment/programming services and inadequate training (Van Voorhis and Brown, 1996). In a more recent review, Byrne and Pattavina (2006) noted similar implementation shortcomings. They identified poor instrument selection processes, lack of validation of imported instruments (only 30% of imported instruments were validated on the local population), poor training, low levels of inter-rater reliability and a lack of quality control/quality assurance (QA/QC) mechanisms (Byrne and Pattavina, 2006). Training is perhaps the largest shortcoming in the implementation of RNA systems. This is illustrated well by this vignette. In a consulting engagement by this author, a group of PPOs were interviewed regarding implementation of EBPs. In discussing the RNA instrument that had been adopted, they said, “we don’t understand it, we don’t trust it and we won’t use it.” It was obvious that the training provided was grossly inadequate and the PPOs (a highly experienced and motivated group) did not receive the information they needed. Without adequate understanding of the RNA technology—how it is developed, how it works, its strengths and limitations and how it assists the PPO—this group was understandably unwilling to accept and use the RNA. All too often, training focuses primarily, if not totally, on how to score the instrument without providing the necessary background on the theory and rationale of actuarial risk assessment, information on how the tools are developed, actuarial vs. clinical judgment, limitations, and other 39

William D. Burrell

materials to establish a knowledge base on which to build (Harris, 2006). Given the departure in practice that actuarial instruments represent for PPOs, this type of information is essential to help staff understand and see the value of RNA in practice. Failure to train staff adequately will undermine the integrity of the RNA, reducing quality and accuracy of assessments and contributing further to the skepticism of PPOs about RNA (Flores et al., 2006). Inadequate training leads to system level problems. “Misapplication of risk assessment often contributes to their low predictive ability” (Gottfredson and Moriarty, 2006a, p. 179). Low predictive ability will further undermine staff views of the value added by RNA. An important tool for supporting implementation is a performance management system. Simple but powerful phrases like “what gets measured matters” send a critical message. If an agency begins to collect data on RNA and then acts on that data, it is a strategic priority for the agency and staff will pay attention. The performance management system is the backbone of Quality Control/Quality Assurance (QA/QC) efforts, monitoring staff performance, providing feedback and documenting outcomes. RNA is widely acknowledged to be a critical element of effective community supervision practices. Many models and instruments are available and are widely in use. Much experience has been accumulated, but the potential remains unrealized. The responsibility for much of this shortfall must be laid at the feet of the field’s leadership. The disappointing level of RNA implementation in probation and parole is due in large measure to the quality of leadership.

Recommendations While the list of problems and issues with the implementation of RNA is long and challenging, it is not insurmountable. Most of the issues described above are not structural but rather managerial in nature. They are almost exclusively management problems that can be resolved. The major exception is the problem of inadequate resources. When faced with inadequate resources, it can be useful to mount a pilot project to demonstrate feasibility and results. Positive outcomes can build support for reallocation of resources to allow full implementation and can also be the basis for pursuing additional resources. As noted above, a number of probation and parole agencies have completed successful implementation of comprehensive EBP models.Their experiences are well documented in the academic and professional literature and in reports of government agency and public policy organizations (See Appendix 2). The growing body of research and scholarship on the science of implementation, also known as technology transfer, provides a rich body of knowledge on how to accomplish program implementation and large scale organizational change.

Implementation Implementing large scale organizational change such as RNA and EBP is not an easy task. Summarizing the issue, implementation scholar Dean Fixen and his colleagues state “Implementation is a decidedly complex endeavor, more complex than policies, programs, procedures, techniques or technologies that are the subject of the implementation efforts.” (Fixen et al., 2005). It is a difficult task, but not impossible. Breaking it down into its component parts reveals a series of manageable tasks. While some tasks may be unfamiliar, none are beyond the ability of community corrections agencies and staff to accomplish. They may need some help, but fortunately a robust body of literature and experience has developed and can be tapped for assistance. The importance of doing implementation well is significant. Poor implementation can make things worse and add to the confusion, doubt and uncertainty, leading to a lack of support among staff (Haas and ­DeTardo-Bora, 2009).

40

Risk and Needs Assessment in Probation and Parole

Detailed guidance on effective implementation of RNA is beyond the scope of this chapter. Some brief comments are warranted. It is important to address implementation comprehensively. At a minimum, the strategy should address seven major topic areas: executive leadership, managing implementation, policies and procedures, risk and need assessment instruments, supervisors, implementation and sustainability. Appendix III contains specific recommendation for each topic area. Appendix II lists resources on implementation.

Research to Advance the State of Practice The challenge of RNA to probation and parole is more one of accomplishing successful organizational change than anything else. Research on organizational change and implementation are two areas where substantial potential for benefit exists. Bridging the gap between research and practitioners remains a huge need, and the technology transfer research is critical. At the line level, developing more effective training models to increase understanding of ­actuarial assessment techniques and the relationship between professional judgment and actuarial models would be useful. The concept of the instrument as an augmentation to human judgment seems to hold some potential for ways to integrate the two aspects of decision-making. Given the pressures of excessive workloads and static or shrinking resources, research into the optimal workload model, tied to improved outcomes, could be useful in building support for increased resources.

Conclusion Risk and need assessment is a critically important component of contemporary practice for community corrections. Sound assessments form the foundation of correctional intervention and community supervision. In their 40-year history, RNA has become commonplace in probation and parole but implementation is uneven and often inadequate. This results in missed opportunities for effective intervention, unrealized potential for effectiveness and efficiency and ultimately lost public value in the form of improved community safety and increased offender integration. This unrealized potential is fundamentally a management problem. Successful implementation of RNA or any other EBP initiative requires three things: first, an effective program model, next, effective implementation strategies, and finally, knowledgeable leadership over the long term. (­Burrell and Rhine, 2013) Reviewing the field, it is clear that the first element has been satisfied; there is a robust body of empirically supported strategies and programs for community corrections and the field is generally well aware of it. On the latter two elements, the picture is less positive. Knowledge about effective implementation is growing, but the awareness in the field needs to increase, along with the skills to apply the knowledge. Lastly, the elements of effective leadership are well known, but not widely enough practiced. This mixed message contains a message of hope for probation and parole and its leadership. Remedying the state of RNA poses a number of challenges, but as noted earlier, none of them are insurmountable or beyond the ability of committed managers and leaders to successfully meet. Information and resources are available to assist; committed leaders need to take the first step.

Note 1 The abbreviation PPO will be used to refer to the many titles used in this field, including probation officer, parole officer, parole agent, probation and parole officer, juvenile probation officer and community supervision officer to note the most common.

41

William D. Burrell

Appendix I Elements of a Comprehensive Assessment and Case Management System   1 Actuarial Assessment Instruments Developed on target population using appropriate methodology Construction and validation samples   2 Targeted for specific population and decision point Population: Adult, juvenile, sex offenders, violent offenders Decision points: Intake (juvenile), pretrial release, diversion, sentencing, supervision, parole release, reentry   3   4   5   6   7

Validated on local population if developed elsewhere Periodic revalidation of instruments—every five years Assess risk, needs and strengths/protective factors Use static and dynamic factors Screening instrument to identify low risk cases Save resources for full assessment on higher risk cases

  8 Professional Override Written policy Supervisory review and approval of overrides Monitoring and feedback   9 Periodic reassessment of risk and needs Suggested frequency: six months for adult, three months for juveniles 10 Dynamic case plans driven by criminogenic factors Regular updates linked to reassessment 11 Quality Assurance/Quality Control Performance monitoring and feedback to staff 12 Comprehensive training program Instrument theory and rationale, development process Skill development and practice Booster training as needed Regular refresher training 13 Integration and alignment with agency mission, policies and procedures 14 Staff involvement in: Instrument selection Policy and procedure development Implementation Training Monitoring 15 Regular reporting of supervision outcomes tied to risk and need scores

42

Risk and Needs Assessment in Probation and Parole

Appendix II Implementation Resources Brennan, T. (1999). Implementing organizational change in criminal justice: some lessons from jail classification systems. Corrections Management Quarterly 3(2), 11-27. Burrell, W. D. (2012). Risk and community corrections. in Burrell, W. D. Community Corrections Management. Kingston, NJ: Civic Research Institute. Chapter 26. Clawson, E. & Guevara, M. (2011). Putting the pieces together: practical strategies for implementing evidence-based practices. Washington, DC: National Institute of Corrections. Fabelo, T., Nagy, G. & Prins, S. (2011). A ten step guide to reforming probation departments to reduce recidivism. New York: Council of State Governments Justice Center. Kreamer, S. J. (2004). Quality assurance and training in offender assessment. Topics in Community ­Corrections—­ 2004. Washington, DC: National Institute of Corrections. 13-19. Serin, R. C. & Lowenkamp, C. T. (2015). Selecting and using risk and need assessments. Alexandria, VA: National Drug Court Institute. Taxman, F. S. (2013). 7 keys to ‘make EBPs stick’: lessons from the field. Federal Probation. 77(2), 76-86. Taxman, F. S. Belenko, S. (2012). Implementing evidence-based practices in community corrections and addiction treatment. New York: Springer. Taxman, F. S., Shepardson, E. S., Delano, J., Mitchell, S. & Byrne, J. M. (2006). Tools of the trade: a guide to incorporating science into practice. Washington, DC: National Institute of Corrections and Maryland Department of Public Safety and Correctional Services. Vincent, G. M., Guy, L.S. & Grisso, T. (2012). Risk assessment in juvenile justice: a guidebook for implementation. Available at: http://www.modelsforchange.net/publications/346.

Appendix III Implementation Topic Areas and Recommendations The following is a list of implementation topics and related recommendations compiled from a variety of sources including references marked with an asterisk (*) on the reference list, as well as the author’s experience. Additional implementation assistance resources are listed in Appendix II. The recommendations are organized into categories based on subject. They include:

Executive Leadership   1 Develop in-depth knowledge of RNA and related evidence-based practices within the agency executive team. These individuals will have to communicate about RNA in an informed and credible manner with internal staff and external stakeholders.   2 Build top level commitment to RNA implementation within the organization. All managers and supervisors will have to be on board. Develop RNA champions to support implementation.   3 Conduct an orientation session(s) for all staff to introduce the RNA implementation. Provide for questions and answers about the project. Use this as an opportunity to recruit staff to be involved in various aspects of the implementation.   4 Build external stakeholder and staff buy-in, support and commitment.   5 Work to build positive organizational culture that fosters the adoption and institutionalization of RNA specifically and EBPs more generally. Consider administering organizational climate/culture survey and/or a change readiness survey to determine the agency’s readiness for change. If a survey is done, take action to address the findings and work to build a supportive culture before implementation begins in earnest.   6 Communicate relentlessly about the project, as it is developing and as it rolls out. Keep staff aware of progress, celebrate accomplishments and recognize contributions. Create positive messages.

43

William D. Burrell

Managing Implementation   7 Getting ready for change.Take the time to analyze the agency, determine needs for implementation and identify key actors and responsibilities.Work to secure the necessary resources. Allow adequate time for implementation—many of the most successful examples of EBP implementation in community corrections took several years to complete. Be sure that external stakeholders such as judges, county commissioners, state legislators and others are aware of the timeframes.   8 Appoint a change agent who will serve as the implementation project manager. This person should have credibility with staff, sufficient knowledge of RNA and the authority to hold staff accountable and ensure that things get done, on time and as intended.   9 Secure and allocate sufficient resources for implementation and operations. Provide an adequate level of resources to enable PPOs to conduct quality assessments. This includes time to conduct the assessment and access to the necessary information. 10 Consider the possibility of conducting a pilot project, particularly if it is a large agency. The pilot will allow road-testing of training, policies and procedures and other aspects prior to full implementation.

Policies and Procedures 11 Consider forming a partnership with an outside entity to assist with implementation. This could be a consultant, a researcher or an academic with experience and skills relevant to the project. The Bureau of Justice Assistance for Smart Supervision grants for probation and parole agencies require a partnership with a research or academic organization. 12 Create opportunities for meaningful staff involvement in implementation of RNA. This can include involvement in planning, instrument selection, validation, revisions of policies and procedures, design and implementation of training, monitoring implementation and communications.The greater the degree of staff involvement, the greater their knowledge and buy-in to the process. 13 Be prepared for challenges and resistance. People and organizations naturally resist change. Staff with change fatigue from too many initiatives may be ambivalent about the latest “new thing”. Don’t confuse ambivalence with resistance. 14 Acknowledge and address staff concerns as they are raised. Create mechanisms for people to submit questions and raise issues, confidentially if necessary. Respond to all inquiries in a professional manner. Share information and responses with all staff, as others may have some of the same questions or concerns. 15 Revise policies and procedures to integrate the RNA into case processing and case management routines of the agency. This will help to ensure that staff complete the RNA as required by making the RNA an integral part of the agency’s decision-making protocol. 16 Link the RNA to case plans, supervision strategies and service/treatment providers.This helps to demonstrate the importance of RNA to processes other than just assessment and will help to connect RNA to outcomes. 17 Develop a comprehensive professional override system.This includes guidelines for appropriate overrides, mandatory supervisory review and sign-off on all override requests, monitoring of frequency, direction (increase or decrease), nature and results of overrides and feedback to staff. 18 Use implementation as an opportunity for work process redesign of the supervision component. Once the “hood is open” on agency processes, take advantage of that to reexamine, reengineer and revise policies and procedures that may be ripe for change. 19 Strive to create a user-friendly RNA system. If possible, automate the process to reduce paperwork, duplicate data entry and increase quality and reliability 44

Risk and Needs Assessment in Probation and Parole

Risk and Need Assessment Instruments 20 Select instrument(s) that are well-designed, tested and documented, appropriate for the target populations and decision points, have comprehensive training materials and curricula available, and have been successfully implemented in other jurisdictions. 21 Conduct validation testing on the instrument(s) if they are imported from outside the agency. Ensure that they are valid and reliable for the target population. If necessary, make modifications, consulting with knowledgeable experts in the field.

Supervisors 22 Develop a robust first line supervisor capacity. These supervisors will be responsible for a range of activities—including training, coaching, monitoring, feedback, override approvals, case reviews—which they may not have done in the past and for which they may not be trained or prepared.This capacity should be developed in advance of RNA implementation by line PPOs. When they have questions, the PPOs will turn to their supervisor, and that supervisor must be prepared to answer the questions correctly and provide the necessary support. Nothing will undermine implementation more quickly than having ill-prepared and ill-equipped supervisors giving blank stares or incorrect information to their staff.

Implementation 23 Develop and implement a comprehensive training program. This should include information on the theory and rationale for actuarial RNA, discussions of clinical vs. actuarial decision-making, professional overrides, limitations of actuarial models, scoring, inter-rater reliability and interpretation of results. Trainees should complete practice cases and have the opportunity to discuss and analyze the cases. Supervisors should monitor staff performance and provide feedback and coaching as necessary. Booster sessions should be available and regular refresher training should also be conducted to keep skills well honed. 24 Create a Quality Control/Quality Assurance (QC/QA) system to monitor staff performance, compliance with policies and assessment quality.

Sustainability 25 Build or enhance the performance feedback system to include information on RNA (use, scoring distribution, client outcomes) and use the information in agency management and resource allocation decisions. Use the results to monitor performance and provide feedback. Staff will notice that the data is being used and that will reinforce the facts that RNA is here to stay. 26 Build for sustainability of RNA by routinely reporting on assessment and outcome data, using data for management decisions, monitoring measures like inter-rater reliability, overrides, by holding regular booster and refresher training sessions, monitoring treatment provider outcomes by RNA categories and providing feedback to staff, and using RNA and outcome data within recruiting, negotiating and contracting with treatment and service providers.

References Note: References marked with an asterisk (*) contain helpful information about implementation. Alexander, M., Whitley, B., & Bersch, C. (2014). Driving evidence-based supervision to the next level: Using PCRA, ‘drivers’ and effective supervision techniques. Federal Probation. 78(3) 2–8. Allen, F. (1981). The decline of the rehabilitative ideal. New Haven:Yale University Press.

45

William D. Burrell Andrews, D. A. & Bonta, J. (2010). The psychology of criminal conduct. 5th ed. New Providence, NJ: Matthew Bender & Co. Andrews, D. A., Bonta, J., & Hoge, R. (1990). Classification for effective rehabilitation: rediscovering psychology. Criminal Justice and Behavior. 17(1), 19–52. Andrews, D. A., Zinger, I., Hoge, R. D., Bonta, J., Gendreau, P., & Cullen, F. T. (1990). Does correctional treatment work? A clinically relevant and psychologically informed meta-analysis. Criminology, 28(3), 369–404. Baird, C., Heinz, R. C., & Bemus, B. J. (1979). The Wisconsin Case Classification/Staff Deployment Project Report #14: Two year-follow-up report. Madison, WI: Department of Health and Social Services, Bureau of ­Community Corrections. *Ballucci, D. (2012). Subverting and negotiating risk assessment: a case study of the LSI in a Canadian youth custody facility. Canadian Journal of Criminology and Criminal Justice. 54(2), 203–228. Barry-Jester, A. M., Casselman, B., & Goldstein, D. (2015). The new science of sentencing. The Marshall Project. https://www.themarshallproject.org/2015/08/04/the-new-science-of-sentencing?utm_medium= email&utm_campaign=newsletter&utm_source=opening-statement&utm_term=newsletter-20150804-240. Bohnstedt, M. & Geiser, S. (1979). Classification instruments for criminal justice decisions: Probation/parole level of supervision sourcebook. Washington, DC: National Institute of Corrections. Bonta, J. (1996). Risk-needs assessment and treatment. In Harland, A. T. (Ed.), Choosing correctional options that work. Thousand Oaks, CA: Sage Publications, 18–32. Bonta, J. & Andrews, D. A. (2007). Risk-need responsivity model for offender assessment and rehabilitation. Ottawa, Ontario: Public Safety Canada. Bonta, J., Rugge, T., Scott, T., Bourgon, G., & Yessine, A. K. (2008). Exploring the black box of community supervision. Journal of Offender Rehabilitation 47(3), 248–270. *Bridges, W. (2003). Managing transitions: making the most of change. 2nd ed. Cambridge, MA: Perseus Publishing. *Brumbaugh, S. & Steffy, D. M. (2005). The importance of constructing and validating risk assessment instruments in community corrections Justice Research and Policy 7(2), 57–85. Bureau of Justice Statistics. (2015). Community corrections statistics. Available at: http://www.bjs.gov/index. cfm?ty=tp&tid=15. Burke, P. B. (1990). Classification and case management in probation and parole: Don’t shoot the messenger. Perspectives 14(3), 37–42. *Burrell, W. D. (2012). Implementation: the Achilles heel of evidence-based practices. In Burrell, W. D. Community Corrections Management. Kingston, NJ: Civic Research Institute. Chapter 22. *Burrell, W. D. & Rhine, E. E. (2013). Implementing evidence-based practices in community corrections: a review essay Justice Research and Policy 15(1), 143–157. *Byrne, James M. & Pattavina, A. (2006). “Assessing the role of clinical and actuarial risk assessment in an ­evidence-based community corrections system: issues to consider.” Federal Probation. 70(2), 64–67. Darlington, R. B. (n.d.). Combining human judgment and multiple regression. http://www3.psych.cornell.edu/ Darlington/predict.htm. Davenport,T. H. & Kirby, J. (2015). Beyond automation: strategies for remaining gainfully employed in the era of very smart machines. Harvard Business Review 73(6), 58–63. Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science 31, 1668–1673. Feeley, M. M. & Simon, J. (1992). The new penology: notes on the emerging strategy of corrections and its implications. Criminology 30(4), 449–474. *Ferguson, J. L. (2002). Putting the “what works” research into practice Criminal Justice and Behavior 29(4), 472–492. Fitzgibbon, D. W. M. (2007). Risk analysis and the new practitioner. Punishment and Society 9(1), 87–97. *Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: a synthesis of the literature. Chapel Hill, NC: University of Carolina, Chapel Hill, National Implementation Research Network. http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/pdf/Monograph_full.pdf. *Flores, A. W., Lowenkamp, C. T., Holsinger, A. M., & Latessa, E. J. (2006). Predicting outcome with the Level of Service Inventory-Revised:The importance of implementation integrity. Journal of Criminal Justice 34(5), 523–529. Gerlinger, J. & Turner, S. F. (2015). California’s Public Safety Realignment: correctional policy based on stakes rather than risk. Criminal Justice Policy Review 26(8), 805–827. Gottfredson, D. M. (1967). Assessment and prediction methods in crime and delinquency in Task Force Report: Juvenile Delinquency and Youth Crime. Washington, DC: The President’s Commission on Law Enforcement and the Administration of Justice. 171–187. Gottfredson, D. M., Cosgrove, C.A., Wilkins, L. T., Wallerstein, J., & Rauh, C. (1978). Classification for parole decision policy. Washington, DC: National Institute of Law Enforcement and Criminal Justice.

46

Risk and Needs Assessment in Probation and Parole Gottfredson, D. M. & Tonry, M. J. (1987). Prediction and classification: criminal justice decision making. Chicago, Il: University of Chicago Press. Gottfredson, S. D. (1987). Prediction: An overview of the issues in Gottfredson, D. M. & Tonry, M. J. (1987) Prediction and classification: criminal justice decision making. Chicago: University of Chicago Press. 22–51. Gottfredson, S. D. & Gottfredson, D. M. (1993).The long-term predictive utility of the Base Expectancy Score. Howard Journal 32(4), 276–290. Gottfredson, S. D. & Moriarty, L. J. (2006a). Statistical risk assessment: old problems and new applications. Crime and Delinquency 52(1), 178–200. Gottfredson, S. D. & Moriarty, L. J. (2006b). Clinical versus actuarial judgments in criminal justice decisions: should one replace the other? Federal Probation 70(2), 15–18. Government Accounting Office. (1976). State and county probation: systems in crisis. Washington, DC: author. *Guy, L. S., Nelson, R. J., Fusco-Morin, S. L., & Vincent, G. M. (2014). What do juvenile probation officers think of the SAVRY and YLS/CMI for case management and do they use the instruments properly? International Journal of Forensic Mental Health 13(4), 227–241. *Haas, S. M. & DeTardo-Bora, K. A. (2009). Inmate reentry and the utility of the LSI-R in case planning. Corrections Compendium Spring, 11–17, 49–53. Harcourt, B. E. (2007). Against prediction: profiling, policing, and punishing in an actuarial age. Chicago, Il: University of Chicago Press. *Harris, P. M. (2006). What community supervision officers need to know about actuarial risk assessment and clinical judgment. Federal Probation 70(2), 8–14. *Harris, P. & Smith, S. (1996). Developing community corrections: an implementation perspective in Harland, A. T. (Ed.) Choosing correctional options that work. Thousand Oaks, CA: Sage Publications, 183–222. Hilton, N. Z., Harris, G.T., & Rice, M. E. (2006). Sixty-six years of research on clinical versus actuarial prediction of violence. The Counseling Psychologist 34(3), 400–409. Hoffman, P. B. & Beck, J. L. (1974). Parole decision-making: a Salient Factor Score. Journal of Criminal Justice 2(3), 195–206. *Holsinger, A. M., Lurigio, A. J., & Latessa, E. J. (2001). Practitioners’ guide to understanding the basis of assessing offender risk. Federal Probation 65(1), 46–50. Horsefield, A. (2003). Risk assessment: who needs it? Probation Journal 50(4), 374–379. Jannetta, J., Elderbroom, B., Solomon, A., Cahill, M., Parthasarathy, B., & Burrell,W. D. (2009). An evolving field: findings from the 2008 parole practices survey. Washington, DC: The Urban Institute. Jannetta, J. & Horvath, A. (2011). Surveying the field: state-level findings from the 2008 parole practices survey. Washington, DC: The Urban Institute. Katzive, M. C. (1968). New areas for bail reform. New York:Vera Institute for Justice. http://www.vera.org/sites/ default/files/resources/downloads/1497.pdf. *Latessa, E. J. (2003). Best practices of classification and assessment. Journal of Community Corrections. 13(2), 4–6, 25–27. *Latessa, E. J. & Lovins, B. (2010). The role of offender risk assessment: a policy maker guide. Victims and Offenders 5(3), 203–219. Lipsky, M. (1980). Street level bureaucracy: dilemmas of individuals in public services. New York: Russell Sage Foundation. Lynch, M. (1998).Waste managers? the new penology, crime fighting, and parole agent identity. Law and Society Review 32(4), 839–869. Lyons, P. & Jermstad,T. (2013). Civil liabilities and other legal issues for probation/parole officers and supervisors. 4th ed. Washington, DC: National Institute of Corrections. Available at: http://nicic.gov/library/027037. Martinson, R. (1974). “What works? questions and answers about prison reform.” The Public Interest. 35, 22–45. Maupin, J. R. (1993). Risk classification system and the provision of juvenile aftercare. Crime and Delinquency 39(1), 90–105. MHS Assessments (2015). 2015 Catalog. North Tonawanda, NY: author. Miller, G. A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. The Psychological Review 63(2), 81–97. *Miller, J. & Maloney, C. (2013). Practitioner compliance with risk/needs assessment tools: a theoretical and empirical assessment. Criminal Justice and Behavior 40(7), 716–736. Mulvey, E. P. (2005). Risk assessment in juvenile justice policy and practice. in Heilbrun, K., Sevin Goldstein, N. E & Redding, R. eds. Juvenile delinquency: prevention, assessment and intervention. New York, NY: Oxford University Press. 209–231. Mulvey, E. P. & Iselin, A. R. (2008). Improving professional judgments of risk and amenability in juvenile justice. The Future of Children 18(2), 35–57. National Institute of Corrections. (1980). Classification in probation and parole: a model systems approach. Washington, DC: author. Mimeo.

47

William D. Burrell Northpointe Institute for Public Management http://www.northpointeinc.com/risk-needs-assessment. Oleson, J. C.,Van Benschoten, S. W., Robinson, C. R., & Lowenkamp, C. T. (2011). Training to see risk: measuring the accuracy of clinical and actuarial risk assessments among Federal Probation Officers. Federal Probation 75(2), 52–56. Perdoni, M. L., Taxman, F. S., & Fletcher, B. W. (2008). Treating offenders in the community: an overlooked population and a lost public health and public safety opportunity. Perspectives 32(2), 46–53. *Perrault, R. T., Paiva-Salisbury, M., & Vincent, G. M. (2012). Probation officers’ perceptions of youths’ risk of reoffending and the use of risk assessment in case management. Behavioral Sciences and the Law 30(4), 487–505. Peterson-Badali, M., Skilling, T., & Haqanee, Z. (2013). Examining implementation of risk assessment in case management for youth in the justice system. Criminal Justice and Behavior 42(3), 304–320. Reichman, N. (1986). Managing crime risks: toward an insurance-based model of social control. Research in Law, Deviance and Social Control 8, 151–172. Rhine, E. E. (2012).The present status and future prospects of parole boards and parole supervision. in Petersilia, J. & Reitz, K. R. The Oxford handbook of sentencing and corrections. New York, NY: Oxford University Press. 627–656. *Rhine, E. E., Mawhorr, T. L., & Parks, E. C. (2006). “Implementation: the bane of effective corrections programs” Criminology and Public Policy 5(2), 347–358. Rhine, E. E., Smith, W. R., & Jackson, R. W. (1991). Paroling authorities: recent history and current practice. Laurel, MD: American Correctional Association. Robinson, G. (2002). Exploring risk management in probation practice. Punishment and Society 4(1), 5–25. Robinson, G. (2003). Implementing OASys: Lessons from research into LSI-R and ACE. Probation Journal 50(1), 30–40. Robinson, G. & McNeill, F. (2008). Exploring the dynamics of compliance with community penalties. Theoretical Criminology 12(4), 431–449. Schein, E. H. (2010). Organizational culture and leadership, 3rd ed. San Francisco: Jossey-Bass. *Schneider, A. L., Ervin, L., & Snyder-Joy, Z. (1996). Further exploration of the flight from discretion: the role of risk/needs instruments in probation supervision decisions. Journal of Criminal Justice. 24(2), 109–121. Schwalbe, C. (2004). Re-visioning risk assessment for human service decision-making. Children and Youth Services Review 26(6), 561–576. Schwalbe, C. S. (2008) A meta-analysis of juvenile justice risk assessment instruments: predictive validity by gender. Criminal Justice and Behavior 35(11), 1367–1381. Silver, E. & Miller, L. (2002). A cautionary note on the use of actuarial risk assessment tools for social control. Crime and Delinquency 48(1), 138–161. Slobogin, C. (2012). Risk assessment and risk management in juvenile justice. Criminal Justice 27(10), 10–18, 25. *Solomon, A. L., Osborne, J.W.L., Winterfield, L., Elderbroom, B., Burke, P., Stroker, R. P., Rhine, E. E., & Burrell, W. D. (2008). Putting public safety first: 13 parole supervision strategies to enhance reentry outcomes. ­Washington, DC: Urban Institute. *Taxman, F. S. & Belenko, S. (2012). Implementing evidence-based practices in community corrections and addiction treatment. New York, NY: Springer. Van Voorhis, P. & Brown, K. (1996). Risk classification in the 1990s. Report submitted to the National Institute of Corrections. Mimeo. Available at: http://nicic.gov/library/013243. * Vigilione, J., Rudes, D.S., & Taxman, F. S. (2015). Misalignment in supervision: implementing risk/needs assessment instruments in probation. Criminal Justice and Behavior 42(3), 263–285. *Vincent, G. M., Guy, L. S., Gerhenson, B. G., & McCabe, P. (2012). Does risk assessment make a difference? results of implementing the SAVRY in juvenile probation. Behavioral Sciences and the Law 30(4), 384–405. *Vincent, G. M., Guy, L. S., & Grisso, T. (2012). Risk assessment in juvenile justice: a guidebook for implementation. Available at: http://www.modelsforchange.net/publications/346. Wormith, J. S., Hogg, S. M., & Guzzo, L. (2015). The predictive validity of the LS/CMI with Aboriginal offenders in Canada. Criminal Justice and Behavior 42(5), 482–508. *Young, D., Moline, K., Farrell, J., & Bierie, D. (2006). Best implementation practices: disseminating new assessment technologies in a juvenile justice agency. Crime and Delinquency 52(1), 135–158.

48

3 ThE RESEArch NA DIrEcTOr PErSpEcTIvE ON ThE DESIGN, IMpLEMENTATION, AND IMpAcT OF RISK ASSESSMENT AND OFFENDEr CLASSIFIcATION SYSTEMS IN USA PrISONS A National Survey James M. Byrne1 and Amy Dezember2 Introduction and Overview Several challenges exist for corrections managers across the United States as they identify and implement “best practices” in the classification, treatment, and control of the prison population under the authority of the state. There have been several national commissions formed over the past decade to investigate the causes and consequences of incarceration, the nature and extent of prison violence, and most recently, the overuse of segregation to control offenders. As a result of the recommendations of these commissions, state corrections managers are being asked to critically examine their correctional management and control strategies, focusing primarily on the development of policies and procedures that make prisons safer, support positive offender change, and prepare individuals for reentry to the community. These recommendations for changes in the area of prison classification, control, treatment, and reentry can be viewed as essential features of a new treatment-focused crime control strategy. Corrections managers and public officials now recognize the inherent limitations of the four decade long p­ unishment-focused crime control strategy, referred to by some commentators as the “great prison experiment” (Byrne, 2013; Clear and Frost, 2014). In recent years, policy makers across the country have been rethinking their approach to crime, and developing strategies designed to expand sentencing options that serve to reduce the use of incarceration as our primary crime control strategy for drug related crimes. These efforts have not yet resulted in significant overall reductions in the U.S. prison population. The United States still has one of the three largest prison systems in the world. Half of all prisoners worldwide are housed in prisons in the United States, China, and the Soviet Union (Byrne, Pattavina, & Taxman, 2015). In the United States, recent efforts to reduce our reliance on prison as the sanction of choice have certainly slowed the rate of increase, but prison and jail populations in 2014 were still higher than the 2000 totals. In 2014, there were 1,561,500 adult inmates 49

James M. Byrne and Amy Dezember

housed in state prisons across the United States, with an additional 744,600 inmates serving time in our jails. While the state prison population declined slightly in 2014 compared to 2013 (1% decline), the jail population increased slightly (up 1.8%). It is critical to think strategically about how best to manage these offenders, because downsizing strategies—by definition—are not focused on what happens in prison. For offenders sentenced to prison, decisions are made every day on where to house them, how to protect and treat them, how to keep them safe and healthy, and when to release them back to the community. The U.S. currently has over 5,000 adult prisons and jails, each with its own unique features, staffing ratios, design and operational capacity, offender population resource level, and reentry protocols. Another characteristic of prisons to add to this list: prison violence. There is variation in the levels and rates of prison violence and disorder by type of facility (maximum, moderate, minimum security) but the overall level and rates of prison violence are actually low. According to Byrne and Hummer’s review of violence in prison between 1995 and 2005, “A review of the official data on the extent of the prison violence problem (murder, rape, and assault) suggests that the most serious forms of violence are rare in federal and state prisons and that the rate of violence in federal and state prisons is actually slightly on the decline, despite the doubling of our prison population in the last decade (Useem & Piehl, 2006)” (2007, p. 79). With a low overall base rate for prison violence, it is more difficult for classification experts to correctly predict who the likely violent prisoner is, but there are new risk assessment tools designed to make these predictions (see Berk, Kriegler, & Baek, 2006; McGuire, 2016). For example, one forecasting model designed and tested by Berk and colleagues on a sample of California inmates identified the following violent prisoner profile: “The high risk inmates tend to be young individuals with long criminal records, active participants in street and prison gangs, and sentenced to long prison terms” (2006, p. 9). The researchers point out that the predictive accuracy of the models they tested was low, resulting in a high false positives rate (10/1). Questions have also been raised about the kinds of data that can be accessed to develop these forecasting models, including the race or ethnicity of the inmate (Byrne & Hummer, 2007b). According to the Commission on Safety and Abuse in America’s Prisons: Reducing violence among prisoners depends on the decisions corrections administrators make about where to house prisoners and how to supervise them. Perhaps most important are the classifications decisions managers make to ensure that housing units do not contain incompatible individuals or groups of people: informants and those they informed about, repeat and violent offenders and vulnerable potential victims, and others who might clash with violent consequences. And these classifications should not be made on the basis of race or ethnicity, or their proxies. (Johnson v. California, 2005). (2006, p. 29) There has been considerable debate among policymakers regarding the proper management of our prison population, not only in terms of how (or for some categories of offenders, whether) risk assessment should be used to make initial placement decisions, but also regarding how risk assessment can be used to inform prison management strategies designed to proactively reduce violence and further risk of recidivism while in prison and in the community upon release from prison. To gain a better appreciation of how risk assessment is currently employed by corrections managers across the United States, we surveyed the directors of research in state departments of corrections regarding current risk assessment policies and practices. The results of this survey reveal significant variation in how risk is conceptualized, assessed, and incorporated into institutional management and subsequent offender reentry strategies. Recommendations for changes in the design/utilization of risk assessment technology are offered in three areas: (1) the need to redesign risk assessment (and reassessment) instruments using the prediction of violence and

50

The Research Director Perspective

disorder in prison as the sole outcome of interest; (2) the need to develop risk assessment instruments that use a full array of individual and community level risk variables that more accurately predict recidivism during reentry to the community; and (3) the need for prison researchers to advance the use of sound methodologies to ensure that risk assessment tools are meaningful in prison environments.

Research Design The brief Survey of Department of Corrections Research Directors on the Current Status and Future ­Direction of Risk Classification Systems consists of three sections with a total of 32 survey items covering: (1) the current status of risk assessment and classification system in each responding state, (2) the research evaluating the effectiveness of risk systems used, and (3) the future developments in risk classification in corrections (see survey instrument in Appendix A).The web survey was sent out to 50 individuals who were identified as research directors in state-level department of corrections agencies from each state. The respondents were notified that the survey was part of ASC’s Division on Corrections & Sentencing’s upcoming edition of the Handbook on Risk and Needs Assessment: Theory and Practice and that the results would inform the researchers on how departments use risk assessment and classification tools. The survey invitation was emailed out on December 3, 2015 to all 50 potential respondents along with a short description of the survey and an individual link directly to the online survey. They were notified that the survey would take approximately 10 minutes to complete. Two weeks later, the first reminder was sent out on December 17, 2015 asking respondents to complete the short survey and reiterated the description of the study and the value of their response to the study. A third and final email reminder was sent out on January 5, 2016 to all individuals that had not yet completed the survey. Additionally, follow up phone calls were made on January 11–13, 2016 to encourage respondents to fill out the survey. After six weeks in the field, data collection was closed with 64.0% (32 out of 50) of respondents completing all or part of the survey3. Table 3.1 lists the state directors who participated in this survey.

Table 3.1  States Included in the National Survey of Research Directors Responding states Michigan Minnesota Missouri Nevada New Hampshire New York Oklahoma Pennsylvania Rhode Island South Carolina Utah Vermont Virginia Washington Wisconsin Wyoming

Alabama Alaska Arizona Arkansas California Colorado Connecticut Delaware Florida Hawaii Idaho Iowa Kansas Kentucky Louisiana Maine

51

James M. Byrne and Amy Dezember

Findings Current Status of Risk Assessment and Offender Classification System There has been a dramatic change in the policies and procedures used for initial assessment of offenders entering our prison system. Less than 15 years ago, a review of classification systems by James Austin and colleagues (2003) identified the Quay system—also known as the adult internal management system (AIMS)—as the most commonly used internal classification system used in state prisons. However, only one survey respondent indicated they used the Quay system while three others indicated some other hybrid method. The Quay classification system is focused on five personality types, such as inadequate-immature, neurotic-conflicted, unsocialized aggressive or psychopathic, sociable or subcultural offenders, and subcultural-immature offenders (Kratcoski, 1981). The goal is to ensure that the prison units are balanced within these five characteristics to improve safety in prison. However, it appears that many states have invested in a wide range of new classification systems, utilizing a combination of internal expertise and external NIC consultants to develop hybrid and/or new assessment systems. When asked about the primary factors driving placement in minimum, medium, or maximum security housing units, 11 of 20 respondents (55%) indicated that the risk classification score was the primary factor, 4 (20%) noted risk level, 2 (10%) referenced conviction offense, and 1 (5%) simply said available space. The respondents noted a variety of secondary factors that influenced the initial placement decision, including conviction offense, prior institutional conduct, risk level, interpersonal or gang-related issues, and available space. Clearly, there are multiple considerations in play when making the initial security level determination but the tendency is not to use personality traits such as those deployed in the Quay management system.While risk of prison violence is not the main driver at this initial decision point, the majority of respondents still view it as the primary purpose of initial classification, particularly regarding examining prior institutional conduct and interpersonal or gang-related issues. Additionally, we asked the research directors to indicate areas where the initial classification of prisoners upon entry to prison could be improved. Of the 18 research directors who responded to this question, 5 (27.8%) recommended a greater initial focus on offender needs and 2 (11.1%) suggested that appropriate offender treatment programs be identified at this initial assessment point. It appears that for a number of respondents, more attention to a third generation risk and need assessment tool that links risk to needs is important. One research director stated, “We’re using our static risk assessment to determine initial level (intensity) of supervision, and then the DRAOR [Dynamic Risk Assessment for Offender Re-entry] scores modify (increase/decrease) level of supervision as time progresses.” Another research director indicated that the risk assessment used in their state was selected because,“COMPAS is a 4th generation actuarial assessment tool.” Lastly, a research director indicated, “We are in the process of implementing a combined static and dynamic risk assessment, which also factors in correctional events, such as programming, infractions, violations and visiting.”With many research directors selecting 3rd and 4th generation tools, there does appear to be a general acceptance of the notion that there is value in the assessment of both risk and needs for classification and treatment decisions. This points to the need to develop classification systems designed for the dual purpose of violence reduction while in prison and offender change upon release to the community. Many directors highlighted the need for subsequent development and implementation of risk reduction strategies. For a small number of respondents (4) risk reduction was viewed as a function of deterrence-based strategies. Additionally, three respondents indicated that current classification systems could be improved by identifying treatment programs for the individual assessed.This shows that there is some division on how research directors believe that prisons want to use a classification system to fit their needs based on the outcomes they desire. Some institutions may want to have greater emphasis on deterrence and treatment, while others may be more focused on incapacitation and improving safety through reduced prison violence. Based on survey responses, it certainly appears that the tripartite deterrence vs. incapacitation vs. treatment debate is ongoing among state research managers. 52

The Research Director Perspective Table 3.2  Which Risk Assessment Instrument Is Being Used in Your System?

COMPAS ORAS LSI-R/LSCMI Developed own risk tool Other Total No Response

N

%

2 4 6 7 2 21 11

9.5 19.0 28.6 33.3 9.6 100.0

We asked respondents about the specific assessment instruments used in their prison system. Of the 21 state research directors who responded, 7 (33.3%) indicated that their state decided to develop their own risk assessment tool, rather than employ a proprietary risk assessment instrument (see Table 3.2). Two state directors discussed how their state uses a static risk tool that they developed since they feel that the static component is the only important factor for initial classification decisions.These states then use other instruments to measure special dynamic needs such as substance abuse, criminal thinking/ cognitions, or dynamic needs. For example, one director stated,“We use [a static risk screening tool created in house] to assess risk and the LS/CMI to assess need among the higher-risk offenders.” Among the remaining states, a variety of proprietary risk instruments were identified, including the LSI-R/ LSCMI (6 states), ORAS (4), COMPAS (2), and PAI or other dynamic/static assessment tool (2). The reasons given for selecting a particular assessment tool varied, but cost, ease of use, and accuracy were the three most common rationales for adoption.The states that use a short-static risk tool indicated that the research team developed their own instrument to reduce the cost of using the standardized tools. Research directors noted that the proprietary instruments as well as the short static tools were originally designed to predict general recidivism after release or during the period of community supervision. They emphasized a need for validating the use of the risk assessment tools in prison to predict prison violence and disorder. Research directors stressed that many of the current tools have not been adequately scientifically reviewed, which raises concerns regarding their current use.

Research Evaluating the Effectiveness of Risk Assessment and Offender Classification Systems There is a small body of research available for review that allows us to assess the accuracy of risk assessment instruments used in prison settings to predict prison violence (Byrne and ­Hummer, 2007a, 2007b; McGuire, 2016, in press). Perhaps the most widely accepted technique for assessing the accuracy of risk predictions is AUC (area under the curve) analysis. According to McGuire’s (2016) review of available studies conducted since 2000 using this validation t­echnique, our ability to accurately predict prison violence is modest at best: The majority of AUC statistics show a significant improvement over chance, though there is marked variation and the predictive success is modest in most cases. The most widely used scales for risk assessment in other contexts (e.g.VRAG, PCL:R and HCR-20) do not emerge especially well from this set of studies, in some cases having no significant association with physical violence. Better results were found using the Risk Assessment Scale for Prison (RASP) and the Risk Assessment for Violent Nonsexual Victimization (RVNSV). The highest figure shown (0.831) is for a simplified model its authors called the Risk Assessment Scale for Prison-­Reduced Burgess (RASP-RB) in respect of its level of accuracy in predicting serious assaults (­Cunningham, Sorensen,Vigen and Woods, 2011). However, in the study in which that scale was originally developed, the observed AUC was less impressive at 0.687 (­Cunningham and Sorensen, 2006b). (McGuire, 2016, p. 37) 53

James M. Byrne and Amy Dezember Table 3.3  What Type of Validation Was Conducted of the Risk Assessment Tool?

Simple correlation AUC analysis Recidivism study Other* All of the above Total No Response

N

%

4 1 3 5 4 17 15

23.5 5.9 17.6 29.4 23.5 100.0

*Respondents who selected other provided the following responses: multiple regression, norming/validation study, or validation was conducted by an outside consultant (2).

One interesting finding from our survey is that the accuracy of the risk assessment tool—when validated—was determined by conducting research on the predictive accuracy of the instrument after release, instead of during the period of incarceration. That is, the tool was validated on subsequent recidivism in the community during a specific follow-up period as the criterion variable. Prison-specific outcomes, such as infractions or prison violence, were seldom used to validate a risk assessment tool. While developing and validating an accurate risk assessment tool for offenders about to leave prison and reenter the community certainly is an important component of risk management in today’s corrections system, it would seem that this is more appropriate for reentry uses than risk assessment during the period of confinement. In fact, given the importance of prison violence, it would seem that there is a need for developing and validating risk instruments designed to predict violence and disorder in prison rather than the community. For example, such instruments might suit the classification decision regarding which housing unit might minimize security issues in prison settings. Since 7 of 20 respondents indicated that they do not currently use a risk instrument designed to predict prison violence, this underscores the need to rethink the purpose of prison risk classification systems. If there is agreement on purpose, new instruments targeting prison violence can be field tested, and decisions can be made about the timing of these risk assessments and how they can be integrated with risk assessments used during the reentry phase of an offender’s prison experience.

Future Developments in Risk Assessment and Offender Classification in Corrections The majority of survey respondents (13/17 or 76%) indicated that improvements in prison classification procedures would reduce the level and rate of prison violence and disorder. Similarly, most respondents (13/18 or 72%) stated that improvements in reentry classification policies and procedures would reduce recidivism among offenders released to the community. A common sentiment was that risk and need assessments are beneficial for “better release planning, transition services, responding to needs early on in community placement/supervision.” Another respondent recognized the importance of making improvements to the classification process, “If we assume we have valid instruments and if the instruments are used to effectively identify high risk/high need inmates for programs and if we are able to place the inmates in the needed programs and if the programs are effectively delivered, THEN it would be logical to conclude a potential reduction in recidivism.” Many research directors identify reentry classification systems as a separate need from initial classification systems. When we asked respondents to consider what other changes they think would reduce prison violence and/or reduce recidivism upon reentry, the most common answer given was that more 54

The Research Director Perspective

effective treatment programming was needed (12), followed by increased incentives for offender change (10). Other strategies included ongoing performance measurement (9), increased staffing (8), better supervision and deterrence (8), and new technology (6). One respondent indicated a need for “better coordination, data collection and resources for analyses among and between CJ agencies.” Another research director emphasized the need for “greater availability of meaningful and effective evidence based programs.” One respondent stated, The tools currently used to assess risk can be improved, some more than others, but the area in greatest need of improvement involves a stronger focus on implementation science. In short, a greater emphasis needs to be placed on how best to bridge the gap, which is rather large, between what we know (i.e., what works) and what we do. Implementation science doesn’t get much attention within corrections, but when it does it’s usually within the context of program delivery. Research directors recognize the need to use evidence-based programs and improve implementation of such efforts to reduce recidivism and bridge the gap between corrections and successful reentry.

Concluding Comments The results of our survey of research directors in state corrections systems underscore the need for a new approach to prison classification, one that views the primary purpose of initial classification as prison violence risk reduction through accurate risk assessment and evidence-based risk reduction planning, program design, and implementation. A major issue identified by the research directors in our survey is that the goal of using a risk and need assessment tool is unclear.The question becomes: how do we design a prison classification system that minimizes the level of prison violence while maximizing the opportunity for long term offender change upon release to the community? The first step in this direction involves the development and proper validation of a risk assessment instrument that targets prison violence as the outcome of interest. According to the recent review by McGuire (2016), only a small number of jurisdictions currently focus on prison violence, and when they do, they typically target a subgroup of violence—prison sexual violence. The importance of focusing on overall security issues such as any prison violence is that it is tied to the health and well-being of the prison environment. A safe prison environment is important, not only for those confined but also for the correctional staff. Generally, classification systems are designed to address these prison environmental issues by ensuring that the mix of individuals in a housing unit will minimize security problems. Risk and need assessment tools that are used for the purpose that they are developed for are more likely to achieve their possible outcomes. As this survey highlights, prison classification systems are designed with multiple purposes in mind, including offender change upon release to the community. If reduction in recidivism is a critical performance measure for institutional corrections, then it will be necessary to identify the range of static and dynamic risk factors that are linked to recidivism (Via, Dezember, & Taxman, 2016). At the same time, research directors recognized that more research is needed to identify accurate predictors of violence in prison settings. The challenge facing these directors is to develop risk assessment tools that address both outcomes: prison violence reduction and recidivism reduction. Additionally, with only about half of the respondents indicating that they have conducted validation studies of the risk and/or need assessment tools used in their state, the results of this survey indicate that many tools currently should be validated to ensure that the risk and need assessments are accurate and reliable. AUC analysis is one of the better methods to evaluate predictive validity of risk assessment tools, yet only five respondents indicated they used this method when validating their tool. The predictive validity of risk assessment tools is vital to the success of proper placement and 55

James M. Byrne and Amy Dezember

classification within a facility.This remains an outstanding issue for the field, which is to have accurate current tools that can predict the desired outcome. A meta-analysis of 47 studies examined how predictive validity is analyzed and reported in studies of instruments used to assess risk and found many inconsistencies in the AUC methodologies and in how validity is measured (Singh, Desmarais, & Van Dorn, 2013). Often, validity studies are not using the correct outcome measures and thus the results may not be accurate. Relatively new to the field are machine learning techniques to help improve the predictive validity of tools available and reduce the cost of using these tools. But, there are cautionary notes about using machine learning models since it is unclear how much they improve the predictive accuracy of the tool, and on the appropriateness of the predictor variables included in these analyses (Hess & Turner, 2017; Kim & Duwe, 2017; Schwartz et al., 2017; Brennan, 2017). Future research should focus on validating different scales and measures that will predict risk of institutional violence and other more accurate outcome measures for use inside of prisons. The research directors recognize that assessing an individual’s risk level is important to address an individual’s dynamic needs to lower their potential for future violence and offending. Many of the research directors surveyed for this study responded that they currently use a proprietary tool that was adopted for their needs. These assessment tools are taken “off-the-shelf ” and adopted on populations that they were not originally designed to serve (Hamilton et al., 2017). This can cause the users of the tool to inaccurately measure outcomes, resulting in the misclassification of inmates. It is important for risk and need assessment tools to be customized to measure the desired outcome (Hamilton et al., 2017). Part of the customization process is ensuring that the selected tool will best serve the agency’s needs, including the staff who will administer it. In many cases, risk assessment tools are not implemented properly, and are not accurately assessing risk and classification levels (Miller & Trocchio, 2017; Rudes, Viglione, & Meyer, 2017). It is important for future research to address how research directors of departments of corrections can select the best risk assessment tools for their needs, customize them for their specific population, and ensure staff buy in for successful implementation (including higher levels of inter-rater reliability). For those offenders beginning the institutional phase of reentry, the challenge for correctional managers will be to link institution-based risk reduction efforts to a community based outcome: the prediction of recidivism upon return to the community. This effort will necessarily require the utilization of a different risk assessment process, since the outcome of interest will shift from institutional behavior to community protection. Perhaps the most important development in this area is the recognition of the link between individual and community level risk factors. According to a recent review: Until we address the underlying community factors that social ecologists have long argued are associated with crime—including location in high risk neighborhoods, culture, resource availability, jobs, poverty, and a breakdown of informal social control ­mechanisms—even high-quality, resource-rich rehabilitation programs are not likely to result in broad-scale desistance from crime among individual offenders. Unless we design correctional strategies that (1) recognize the link between person environment interactions and recidivism, and then (2) attempt to change both individual offenders and individual communities, we will continue the cycling of these individuals from community to prison to community. (Byrne, 2008, p. 270) The results of our survey of research directors in state correction systems underscore the need for revisiting the approach to prison classification, utilizing a perspective that echoes the need to consider the primary purpose of initial classification as prison violence risk reduction through accurate risk and need assessment. The research directors view classification as being linked to evidence-based risk reduction planning, program design, case planning and implementation. For those 56

The Research Director Perspective

offenders beginning the institutional phase of reentry, the challenge for correctional managers will be to focus on a different outcome: the prediction of recidivism upon return to the community. This effort will necessarily require the utilization of a different risk assessment process, since the outcome of interest will shift from institutional protection to community protection. The research directors clearly indicated that the focus on using validated risk and need assessment tools is an important movement, but the tailoring of these tools for the intended use is critically important. The implementation of the risk and need assessment in prison environments is essentially a work in progress, and one that can benefit from more attention to the goals of classification.

Notes 1 Professor, School of Criminology and Justice Studies, University of Massachusetts, Lowell. 2 Doctoral Student, Center for Advancing Correctional Excellence, George Mason University. 3 The draft survey was pre-tested by two research directors from state-level department of corrections agencies and their feedback was incorporated into the final survey. Since one individual was identified from each state, we received responses from 32 of the state-level department of corrections, while 18 states did not respond to the survey after being contacted four times.

Appendix A: Survey Instrument Name: Agency: Survey of Department of Corrections Research Directors on the Current Status and Future Direction of Risk Classification Systems Section 1: Current Status of Risk Assessment and Classification System in Each State 1

Which internal classification system is used in your state corrections system? (Select one) a Quay Adult Internal Management (AIM) System b Prison Management Classification (PMC) System c Hybrid d Don’t Know e None f Other, please specify: _____________________________

2

What factors drive the initial placement of offenders into maximum, medium, or minimum security prisons? (Primary, select one / Secondary, check all that apply) a Conviction offense b Prior institutional conduct c Risk level d Space available e Interpersonal or gang related issues f Classification score g Don’t know h Other-Specify: _____________________________

3

What is the purpose of the classification system in your jurisdiction? (Primary, select one / Secondary, check all that apply) a Risk assessment to determine level of risk that the person presents in prison b Need/service assessment to determined needed programs c Both risk and need assessment d Screening tool 57

James M. Byrne and Amy Dezember

e Classification f Inform access to treatment g Inform housing placement in prison (e.g., general population, administrative segregation, solitary confinement, etc.) h Case planning i Other-Specify: _____________________________ 4

How can the current classification instrument be improved? (Primary v. secondary) a Assess client’s needs b Identify treatment programs c Need to include deterrence approaches d Change outcomes measures (e.g., shift focus towards treatment and away from recidivism) e Need a tool customized to department needs f Shorten assessment tools g Other, please specify: _____________________________

5

Which risk assessment instrument is being used in your system? (Select one) a COMPAS b ORAS c LSI-R d LSCMI e Wisconsin Risk and Needs f RASP g VRAG h PCL-R i RVNSV j HCR-20 k PCL-R l PAI m Combination of the above (please specify):_____________________ n Developed own risk tool (please describe): ________________ o Other (please specify): _____________________________

6

Why was the risk assessment instrument (or instruments) selected? (Select one) a Cost b Ease of use c Accuracy d All of the above e Other, specify: _____________________________

7

What type of validation was conducted of the risk assessment tool? (Select one) a Simple correlation b AUC analyses c Recidivism study d All of the above e Other, specify: _____________________________

8

Which type of validation study of the risk assessment tool is the most valuable to your organization? (Select one) a Simple correlation b AUC analyses c Recidivism study d Other, specify: _____________________________ 58

The Research Director Perspective

9 What is the dependent variable used in the validation studies of the risk assessment tool? (Select one) a Violent crime in prison b Disciplinary infractions in prison c Recidivism upon release to the community d Other, Specify: _____________________________ 10 Have you conducted an evaluation of whether the risk assessment tool used can be scored reliably? a Yes b No 11 When is the risk assessment tool administered? (Select one) a At intake or diagnostic center only b At intake/diagnostic center and then bi-annually until release c At intake/diagnostic center and then at release or near reentry d Other, specify: _____________________________ 12 Who primarily administers the risk assessment tool? (Select one) a Intake staff b Private provider c Clinical staff d Other, please specify: _____________________________ 13 What issues do you experience when using risk assessment tools? (Check all that apply) a Misclassification of offenders b Too time consuming c Tool is not customized to department needs d Requires too many resources to use properly e Tool is outdated f Tool only addresses risk and does not address needs g Not reliable h Inter-rater reliability concerns i Other, please specify: _____________________________ 14 Which of the following do you use to predict prison violence? (Check one) a Actuarial risk instrument b Special violence instrument such asVRAG,or list the tool:__________________________ c Structured professional judgment tool (e.g. HCR-20) d None e Don’t know f Other, specify: _____________________________ 15 Which of the following types of violence and/or disorder are distinguished in your risk assessment or classification system (Check all that apply):

a b c d e

Inmate on inmate Inmate on staff Staff on inmate Collective violence Intra-personal violence

Risk Assessment

Classification System

☐ ☐ ☐ ☐ ☐

☐ ☐ ☐ ☐ ☐

59

James M. Byrne and Amy Dezember

16 Some systems have developed a separate process for assessing risk upon reentry or transition back into the community. Do you have a separate reentry-based risk assessment, classification system, or discharge planning process? a Yes b No If yes, how is this process different? _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ 17 Upon release, when do you assess an inmate’s risk for re-offending? (Select one) a Within 3 months b Within 6 months c Within 1 year d We do not assess an inmate’s risk for re-offending after release e Other, please specify: _____________________________ 18 If yes, which risk assessment instrument do you use? (Select one) a COMPAS b ORAS c LSI-R d Wisconsin Risk Needs e Combination of the above (please specify): _____________________ f We do not assess an inmate’s risk for re-offending after release g Other (please specify): _____________________________ Any additional comments on current use of assessment tools: _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ Section 2: Research Evaluating the Effectiveness of Risk System 1

Has the overall effectiveness of the prison risk assessment system been evaluated? a Yes b No

2

If yes, how was it evaluated? (Select one) a Internal agency assessment b External review c Other, please specify: _____________________________

3

If yes, what was the outcome of the evaluation? (Check all that apply) a Instrument worked well b Needed to adjust the instrument cut-offs c Reduced some items from the instrument d Other, specify: _____________________________ 60

The Research Director Perspective

4

What is your overall assessment of the current classification system used to initially assess new admissions? (Select one) a We have staffing issues that affect the assessment system b The current instrument is not that useful for normal prison operations c The current procedures were updated and we still have problems of violence d Our procedures need improvement (please explain): _______________________

5

Which of the following changes (if any) in the initial assessment process would you recommend? (Check all that apply) a New assessment instrument (please explain): ______________________ b Validation of existing assessment tool(s) (please explain): _____________________ c More and/or better qualified staff (please explain): ______________________ d Other, please specify: _____________________________ e No changes

6

When are inmates re-assessed in your system? (Select one) a Quarterly b Annually c Ongoing, based on behavior d No re-assessment e Other, please specify: _____________________________

7 How would you rate the effectiveness of the re-assessment process currently in place? (Select one) a Excellent b Good c Average d Poor 8

If you use a separate reentry assessment process, how is the performance of the reentry strategy evaluated in your jurisdiction? (Select one) a Internal agency assessment b External review c We do not use a separate reentry assessment process d Other, please specify: _____________________________

9 If you use a separate reentry assessment process, has the reentry system performance been examined to address the following responsivity factors? (Check all that apply) a Age b Gender c Race d Offense type e Location f We do not use a separate reentry assessment process g Other, please specify: _____________________________ If yes, briefly highlight results: ________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ 61

James M. Byrne and Amy Dezember

Section 3: Future Developments in Risk Classification in Corrections 1

In your view, can the level of violence and disorder in prison be reduced through improved classification procedures? a Yes b No

2

In your view, can we reduce the current rate of recidivism through improved reentry classification procedures (e.g. separate reentry risk assessments, classification systems, or discharge planning)? a Yes b No If yes, please explain how so: _________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________

3

What else can we do to reduce recidivism? (Check all that apply) a Increased staffing (number/quality) b New technology c Increased incentives for offender change d Ongoing performance measurement e More effective supervision and deterrence-based approaches f More effective treatment programming availability g Other, Specify: _____________________________

Any comments you would like to make about the state of risk assessment and its use in prison/ community:         

References Austin, J. (2003). Findings in prison classification and risk assessment. Washington, D.C.: U.S. National Institute of Corrections. Berk, R.A., Kriegler, B., & Baek, J. (2006). Forecasting dangerous inmate misconduct. Berkeley, CA: California Policy Research Center, University of California, Berkeley. Brame, B. (2016). Static Risk Factors and Criminal Recidivism. In F.S. Taxman & (Ed.), Risk and Need ­Assessment:Theory and Practice (pages pending). Abingdon: Routledge. Brennan, T. (2016). An alternative scientific paradigm for criminological risk assessment: Closed or Open ­Systems, or both? In F.S. Taxman (Ed.), Risk and Need Assessment: Theory and Practice (pages pending). ­Abingdon: Routledge. Byrne, J.M. (2008) The social ecology of community corrections. Criminology and Public Policy, 7(4), 263–274.

62

The Research Director Perspective Byrne, J. M. (2013). After the fall: Assessing the impact of the great prison experiment on future crime control policy. Federal Probation, 77(3), 3–14. Byrne, J. M. & Hummer, D. (2007a). Myths and realities of prison violence: A review of the evidence. Victims and Offenders, 2, 77–90. Byrne, J. M. & Hummer, D. (2007b). In search of the “Tossed Salad Man” (and others involved in prison violence): New strategies for predicting and controlling violence in prison. Aggression and Violent Behavior, 12, 531–541. Byrne, J. M. & Hummer, D. (2008). Examining the impact of institutional culture on prison violence and disorder: an evidence-based review. In J. M. Byrne, D. Hummer & F. S. Taxman (eds.), The Culture of Prison Violence, pp. 40–90. Boston, MA: Pearson. Byrne, J. M., Hummer, D., & Taxman, F. S. (eds.) (2008). The Culture of Prison Violence. Boston, MA: Pearson. Byrne, J., Pattavina, A., & Taxman, F. (2015). International trends in prison upsizing and downsizing: In search of evidence of a global rehabilitation revolution. Victims and Offenders: An International Journal of E ­ vidence-based Research, Policy, and Practice, 10(4), 420–451. Clear, T.R. & Frost, N. A. (2014). The Punishment Imperative: The rise and failure of mass incarceration in America. New York, NY: New York University Press. Cunningham, M. D. & Sorensen, J. R. (2006b). Actuarial models for assessing prison violence risk: Revision and extensions of the Risk Assessment Scale for Prison (RASP). Assessment, 13, 253–265. Cunningham, M. D., Sorensen, J. R., & Reidy,T. J. (2005). An actuarial model for assessment of prison violence risk among maximum security inmates. Assessment, 12, 40–49. Cunningham, M. D., Sorensen, J. R., Vigen, M. P., & Woods, S. O. (2011). Correlates and actuarial models of assaulting prison misconduct among violence-predicted capital offenders. Criminal Justice and Behavior, 38, 5–25. Endrass, J., Rossegger, A., Frischknecht, A., Noll,T., & Urbaniok, F. (2008a). Using the Violence Risk Appraisal Guide (VRAG) to predict in-prison aggressive behaviour in a Swiss offender population. International Journal of Offender Therapy and Comparative Criminology, 52, 81–89. Endrass, J., Rossegger, A., Urbaniok, F., Laubacher, A., & Vetter, S. (2008b). Predicting violent infractions in a Swiss state penitentiary: A replications study of the PCL-R in a population of sex and violent offenders. BMC Psychiatry, 8–74. DOI:10.1186/1471-244X-8-74. Gendreau, P., Little, T., & Goggin, C. (1996). A meta-analysis of the predictors of adult offender recidivism: what works! Criminology, 34(4), 575–608. Gibbons & Katzenbach (2006). Confronting confinement. Washington, DC:Vera Institute of Justice, Commission on Safety and Abuse in America’s Prisons. Hamilton, Z., Tollefsbol, E.T., Campagna, M., & van Wormer, J. (2016). Customizing Criminal Justice Assessments. In F.S. Taxman (Ed.), Risk and Need Assessment: Theory and Practice (pages pending). Abingdon: Routledge. Hanson, R. K. & Morton-Bourgon, K. E. (2009). The accuracy of recidivism risk assessments for sexual offenders: a meta-analysis of 118 prediction studies. Psychological Assessment, 21(1), 1–21. Hastings, M. E., Krishnan, S., Tangney, J. P., & Stuewig, J. (2011). Predictive and incremental validity of the ­Violence Risk Appraisal Guide scores with male and female jail inmates. Psychological Assessment, 23, 174–183. Hess, J. & Turner, S.F. (2016). Accuracy of risk assessment in corrections population management: Where’s the value added? In F.S. Taxman (Ed.), Risk and Need Assessment:Theory and Practice (pages pending). Abingdon: Routledge. Kim, K. & Duwe, G. (2016). In F.S.Taxman (Ed.), Risk and Need Assessment:Theory and Practice (pages pending). Abingdon: Routledge. Kratcoski, P. C. (1981). Use of the quay system of Classification. California: Duxbury Press. Kroner, D. G. & Mills, J. F. (2001). The accuracy of five risk appraisal instruments in predicting institutional misconduct and new convictions. Criminal Justice and Behavior, 28, 471–489. Labrecque, R. M., Smith, P., & Wooldredge, J. D. (2014). Creation and validation of an inmate risk assessment for violent, nonsexual victimization. Violence and Victims, 9, 317–333. McGuire, James (2016, in press). Prison violence: A review of research on models and Contributors. United Kingdom, National Offender Management Service (NOMS). Miller, J. & Trocchio, S. (2016). Risk/Need Assessment Tools and the Criminal Justice Bureaucrat: Reconceptualizating the frontline practitioner. In F.S. Taxman (Ed.), Risk and Need Assessment: Theory and Practice (pages pending). Abingdon: Routledge. Rudes, D.S.,Viglione, J., & Meyer, K.S. (2016). Risky Needs: Risk entangled needs in probation supervision. In F.S. Taxman (Ed.), Risk and Need Assessment:Theory and Practice (pages pending). Abingdon: Routledge.

63

James M. Byrne and Amy Dezember Schwartz, I.M.,York, P., Greenwald, M., Ramos-Hernandez, A., & Feeley, L. (2016). Using Predictive Analytics and Machine Learning to Improve the Accuracy and Performance of Juvenile Justice Risk Assessment Instruments: The Florida Case Study. In F.S. Taxman (Ed.), Risk and Need Assessment: Theory and Practice (pages pending). Abingdon: Routledge. Singh, J. P., Desmarais, S. L., & Van Dorn, R. A. (2013). Measurement of predictive validity in violence risk assessment studies: A second-order systematic review. Behavioral Sciences & the Law, 31(1), 55–73. Via, B., Dezember, A., & Taxman, F.S. (2016). Exploring How to Measure Criminogenic Needs: Five Instruments and No Real Answers. In F.S. Taxman & A. Dezember (Eds.), Risk and Need Assessment: Theory and Practice (pages pending). Abingdon: Routledge. Walters, G. D., Duncan, S., & Geyer, M. (2011). Predicting disciplinary adjustment in inmates undergoing forensic evaluation: a direct comparison of the PCL-R and the PAI. Journal of Forensic Psychiatry and Psychology, 14, 382–393. Walters, G. D. & Heilbrun, K. (2010).Violence risk assessment and Facet 4 of the Psychopathy Checklist: Predicting institutional and community aggression in two forensic samples. Assessment, 17, 259–268.

64

PART II

Methodological Issues in Creating and Validating RNA

This page intentionally left blank

4 STATIc RISK FAcTOrS AND CrIMINAL REcIDIvISM Robert Brame1

Introduction Most arrests that occur in the United States each year are among people who have been arrested before (Christensen, 1967). Reams of criminological literature tell us that many, many people who have been involved in the criminal justice system in the past will be involved again at some point in the future (see, for example, Durose, Cooper, and Snyder, 2014). American prison populations and the number of people under correctional supervision both remain near their all-time high levels (Kaeble, Glaze, Tsoutis, and Minton, 2016)—a pattern which increasingly appears to be unsustainable (Clear and Austin, 2009). Although there is uncertainty about why recidivism rates are currently high and have been high for many years, it is also true that American criminal justice policy is increasingly fixated on achieving simultaneous reductions in both prison populations and recidivism statistics that are commonly calculated by criminal justice researchers and practitioners (Gottschalk, 2015). To achieve this twopronged goal, policy makers are looking to quantitative, empirically-based risk assessment tools that will help criminal justice practitioners prospectively distinguish people who have different likelihoods of continued criminal justice system involvement. The idea of using ex ante person-specific information to make predictions about the risk of future offending is not new (Harcourt, 2006; 2015). But with the advent of more complete electronic databases, more powerful computers, a diffusion of statistical modeling expertise, a desire among legislators and state executives to reduce correctional spending, and more treatment and re-entry options for offenders (both in institutions and in the community), the idea of formalized risk assessment has a growing appeal in our time. In this essay, I consider several aspects of the literature on risk assessment in general and so-called static risk factors in particular. I follow the vast majority of the literature in defining static risk factors as variables that measure historical features of people (principally, personal demographic characteristics and information about one’s prior criminal history) that can improve predictions about future offending among people who have offended in the past (recidivism).2 It seems appropriate to note that I am not someone who has much experience or background in the development of static risk-assessment tools. Although I have studied and been interested in patterns of criminal recidivism and crime desistance for many years and have been broadly familiar with the recent rise of formalized risk assessment in criminology, I come to this particular literature from a position of relative inexperience. That said, I have a few observations to offer as I consider how static risk factors and recidivism relate to and connect with each other. 67

Robert Brame

Criminal Recidivism In this section, I overview several key issues related to the study and measurement of new offending among those who have previously been involved in the criminal justice system. Among the issues considered are the contribution of recidivist arrests to the total number of arrests, historical data related to major birth cohort study in Philadelphia, a significant historical example of a recidivism study where people are released from prison and then followed in the community for about 18 years, overlap between key concepts in the criminal recidivism and criminal careers literatures, and an overview of the problem of extrapolating recidivism patterns beyond the circumstances under which they were developed.

First-Time and Recidivist Arrests Most U.S. arrests do not involve people who are being arrested for the very first time. This fact has been understood for about 50 years since Ronald Christensen (1967) first demonstrated it in his work for the Science and Technology Task Force of President Johnson’s Commission on Law Enforcement and Administration. Each year, the FBI’s Uniform Crime Reporting (UCR) program presents arrest counts stratified by age group.Within each age group, some of the arrests involve people who are being arrested for the first time while others involve people who have been arrested in the past. At the earliest ages, most of the arrests are first-time arrests but as people get older Christensen showed (using non-FBI data) that the fraction of arrests that are virgin arrests drops steeply with age.3 While research tells us that most arrests are recidivist arrests, it does not follow that most crimes are the result of recidivism (Gottschalk, 2015; Rosenfeld et al., 2002). It is important to remember that the decision to arrest is the combined product of a criminal act, a victim’s reporting of that crime to the police (unless the police observe it directly), and a clearing of the probable cause evidentiary standards for police to take action. The first part (the criminal act) is the basis for evaluating threats to public safety; the other two parts are reactive in nature and may reflect patterns or biases that have little to do with public safety.

Philadelphia Birth Cohort Study A few years later, Marvin Wolfgang and his colleagues (1972) examined police contacts for criminal offenses among a “birth cohort” of males who were born in Philadelphia in 1945 and who resided continuously in the city between the ages of 10 and 17. Figure 4.1 presents a summary of the contact distribution and shows that a little over a third of the boys had at least one police contact by age 18. Using the Philadelphia data, Blumstein and Moitra (1980:325) showed that the proportion with a second contact conditional on having experienced a first (recidivism) was 0.535 while the third conditional on the second was 0.651. Among those with three contacts, the proportion who went on to have 4 or more contacts was 0.715.These progressively high recidivism rates led to a situation where a relatively small number of people in the original cohort (624 out of 9,944) accounted for about 52% of the total contacts for the entire group by age 18.4 A key issue raised by the Blumstein and Moitra paper, however, is that there may be real limits to our ability to ever prospectively predict who the highest-rate offenders will be—before they do most of their damage. Their study was based on a “shifted” geometric statistical model for the police contact counts. What this means is that once one counts to the third contact, a model that assumes a constant value for p(k + 1 contacts|k contacts) is the same for all k > 3. This simple stochastic process fits the Philadelphia data quite well and stands as a strong null hypothesis against any theory that hypothesizes important differences among people who have more than three contacts. It is akin to the idea of a coin flipping experiment where one flips a coin 10 times and counts the 68

Static Risk Factors and Criminal Recidivism

6,479

5k Number of People

Number of People

6k

4k 3k 2k

200 175 150 125 100 75 50 25 0

624 Chronic Offenders; Each With 5+ Contacts

160 99

83 59

38

38

29

32

39

13

21

13

5

6

7

8

9

10

11

12

13

14

15

>15

5

6

7

8

9

10

11

12

13

14

15

>15

1,609

1k

647

0 0

1

2

344

241

3

4

Number of Police Contacts

Figure 4.1  Police Contacts in Philadelphia Cohort

number of heads. Now suppose we do this experiment thousands of times.There will be variability in the number of heads we get but it would be hard to imagine a search for variables to account for that variability. The lesson from this study is that we always have to be aware of the possibility that differences in behavior among people who commit crimes may have some intrinsic randomness which will defy prediction efforts (see also, Starr, 2014:849).5

1956 Federal Prison Releasees Subsequently, a long-term recidivism study was published by Kitchener, Schmidt, and Glaser (1977). The study was based on “a 10 percent systematic sample of the approximately 10,000 male Federal prisoners released in 1956” (Kitchener et al., 1977:9). There were both short- (1959–60) and long-term (18 years) follow-up data collection efforts to track recidivism failures in the form of parole violations and new federal and state offenses as measured by federal records on a subset of 903 people from the original systematic sample (a small number of cases were deleted because of deportation (N = 46) and unsuccessful FBI record traces; the authors reported that 93.2% of the “nondeported cases” were included in this study). My colleagues and I (Kurlychek, Bushway, and Brame, 2012) did not have access to these raw data but we used a chart in the Kitchener et al. (1977) paper to reconstruct the dataset. The results of this effort were presented in that article and are reproduced here as Figure 4.2. Unlike the aforementioned Christensen (1967) and Wolfgang et al. (1972) studies, the Kitchener analysis was on the same footing as a more traditional contemporary recidivism study. There was a well-defined target population (and sample from that population), a reasonably clear definition of what “recidivism” meant, a consistent follow-up period for the entire sample, and measurements documenting when people were observed to recidivate. The left-hand panel of Figure 4.2 repeats what was presented in the original Kitchener et al. (1977) article. The right-hand panel uses the information in the cumulative chart to estimate a fundamental parameter of interest in recidivism studies—the hazard rate. To a reasonable approximation, the hazard rate is a fraction that is calculated at each time unit in the follow-up period. The numerator of the fraction is the number of people who are observed to fail in that time interval while the denominator is the number of people who had not yet failed prior to the beginning of that time interval. The dominant pattern is one of declining risk of failure as time since release from prison increases. Although there is an increase in failure risk at the beginning of the follow-­ ­up period, that risk attained its maximum in the third year. Many contemporary recidivism studies 69

Robert Brame 0.30

80

0.25

18 Year Follow-Up 18 Year Failure Rate = 63%

60 40

Hazard Rate

Cumulative Percent Failing

100

20

0.20

18 Year Follow-Up

0.15 0.10 0.05 0.00

0 0 5 10 15 Number of Years Since Release from Prison in 1956

0 5 10 15 Number of Years Since Release from Prison in 1956

Figure 4.2  Recidivism of 1956 Federal Prison Releasees Source: Permission to reproduce this chart was obtained from the Publisher (Kurlychek, M. C., Bushway, S. C., & Brame, R. (2012). Long term crime desistance and recidivism patterns: evidence from the Essex County convicted felon study. Criminology, 50:71–103.)

exhibit the “long decline” hazard rate pattern revealed in Figure 4.2 (for some examples of this pattern in other datasets, see Beck and Shipley, 1989; Schmidt and Witte, 1989;Visher et al., 1991; Langan and Levin, 2002; Kurlychek et al., 2006, 2007, 2012; Blumstein and Nakamura, 2009; Durose et al., 2014).

Recidivism and Criminal Careers Coinciding with the development of methods for studying how recidivism experiences play out over time was a larger movement to formalize the study of criminal careers (Blumstein et al., 1986; Blumstein and Cohen, 1987). The criminal career paradigm most fundamentally divides the population into people who participate in offending during some well-defined period of time and people who refrain from participating. Among those who participate, the focus shifts to different dimensions of offending including the frequency, seriousness, and duration of offending. The study of recidivism interfaces with each of these latter three concepts. First, we can consider recidivism in terms of the number of new criminal offending incidents (Brame et al., 2003); alternatively, it is plausible that in many cases someone who returns to offending relatively quickly is offending at a higher rate than someone who returns to offending more slowly. In fact, as Bushway and colleagues (2001) noted, a person could still be an active offender even if a careful look at their offending pattern reveals a declining frequency of offending as he or she makes a gradual transition to a state of desistance. Second, we know that many people who are involved in the criminal justice system have been implicated in less serious crimes; a corollary position is that many people who are returning to the community after being sanctioned for very serious crimes are often rearrested for much less serious crimes. In other words, a high recidivism rate doesn’t imply a high recidivism rate for serious offenses.6 Third, time-to-recidivism distributions can provide useful information for measuring the length and termination of criminal careers. Maltz (1984) and Schmidt and Witte (1989) both highlighted the use of so-called “split-population” parametric statistical models that can be used to distinguish between two groups of people: (1) a group of people whose waiting time to the next offense can be modeled with a probability distribution; and (2) a group of people who have effectively desisted from crime. The usual specification of these parametric waiting time models assumes that if we simply follow people long enough, all of them will eventually fail. As Maltz (1984) has noted, this assumption makes a lot of sense for modeling waiting times for the failure of light bulbs and bridges but less sense for criminal recidivism. In fact, models that allow for a limiting failure rate that is significantly less than 100% usually fit recidivism data much better. 70

Static Risk Factors and Criminal Recidivism

A Parametric Recidivism Model I use the Kitchener data described above to illustrate this point. Because the hazard rate in these data has a clear turning point and a long right-hand tail, I decided to fit a lognormal specification. Letting ti be the waiting time to recidivism or the end of the follow-up period (whichever comes first), F and S are the sets of failures (people who are observed to recidivate within the follow-up period) and survivors (people who are not observed to recidivate within the follow-up period), f(t) is the probability density function for t, and F(t) is the cumulative distribution function for t or, alternatively, the cumulative probability of failure by time t (its complement, 1 − F(t), is the survivor distribution function, S(t)), the log-likelihood function for a survival time model is:

log e (L;q ) =

∑ log [ f (t )] + ∑ log [1 − F(t )] e

i

e

i ∈F

i ∈S

i



Now, the components of the log-likelihood functions for the lognormal and split-population lognormal specifications are presented in Table 4.1. The analysis assumes that loge[t] follows a normal distribution with mean, μ, and standard deviation, σ (so that θ = [μ,σ]). The split-population model also produces an estimate of the limiting failure rate, δ, which will be a proportion (so that θ = [ μ,σ,δ]). As δ → 1.0 the split-population model becomes the same as the standard lognormal specification. Finally, we let Φ(·) denote the standard normal cumulative distribution function. Since the split-population lognormal model includes the lognormal as a special case, we can compare the fit of the two models on the same data set. Table 4.2 presents the parameter estimates from both models fit to the Kitchener federal prison releasee dataset over an 18-year follow-up period. At the cost of 1 parameter, the split-population lognormal model achieves a 129.525 point increase in the log-likelihood function; a likelihood-ratio test tells us that this increase is statistically significant at the 95% confidence level. So, the split-population model would seem to be the preferred specification. The splitting parameter estimate, δ, tells us that with the long 18-year follow-­up period the final failure rate observed in the sample (63.0%) is very close to the long-term Table 4.1  Likelihood Components for Lognormal Survival Time Models Quantity

Lognormal

f(t)

 t −m ) 1 ( i log e  2s 2 ti 2πs 2 

S(t)

Split-Population Lognormal

  t − m  1 − Φ ( log e  i   s  

2

   

log e (d ) + log e  f (ti )

log e 1 − d + d 1 − F (ti )

Table 4.2  Survival Time Models Fit to the Kitchener Data Parameter µ σ δ log e [  ]

Lognormal

Split-Population Lognormal

2.072 1.635

1.007 0.806 0.639 -1820.378

-1949.903

71

Robert Brame 0.30

18-Year Failure Rate = 63%

60

40 Observed Data Lognormal Split-Population Lognormal

20

Observed Data

18-Year Follow-Up

0.25

18-Year Follow-Up

80

Hazard Rate

Cumulative Percent Failing

100

0.20 0.15 Split-Population Lognormal

0.10

Lognormal

0.05

0

0.00 0

5

10

0

15

Number of Years Since Release from Prison in 1956

5

10

15

Number of Years Since Release from Prison in 1956

Figure 4.3  Parametric Failure Time Models Fit to Kitchener Data

limiting failure rate dˆ ≈ 0.639 . Of course, 63.9% is far away from 100% so we are not surprised to see that the split population model provides a better fit. Figure 4.3 provides graphical evidence of the split-population model’s superior fit. Notice that both the lognormal and the split-population lognormal model capture important features of the data (i.e., the turning point and the long decline in the hazard rate that we considered earlier). Nevertheless, it is clear that the split-population lognormal specification approximates the observed data much more closely. The standard lognormal model underpredicts the recidivism rate in the first few years of the follow-up period and after about 12 years, it begins to significantly overpredict.This is because the standard lognormal model requires the cumulative failure rate to eventually approach 100%. The pattern of the Kitchener data is very similar to what we see in other criminological data sets (see, for example, Schmidt and Witte, 1989; Kurlychek et al., 2012) and suggests the importance of carefully choosing a reasonable specification for the recidivism waiting time distribution when calibrating predictive models.7

Observed Heterogeneity in Recidivism Models Schmidt and Witte (1989) expanded on the idea of the split-population model to simultaneously measure the separate effects of covariates on the waiting time to recidivism and on whether people have effectively desisted. Their results suggested that demographic characteristics like sex and race were related to whether people had desisted or not but were not related to the timing of recidivism (Schmidt and Witte, 1989:107–108). This pattern of results echoes a key theme of the criminal careers literature: the idea that factors which are related to one dimension of the career may not be related to all dimensions (Blumstein and Graddy, 1982).8

Different Definitions of Recidivism Another important idea—most clearly conveyed by the BJS recidivism studies of Beck and Shipley (1989), Langan and Levin (2002), and Durose et al. (2014)—is that the operational definition of recidivism has significant implications for the amount of recidivism that one is likely to measure.The recidivism rates in the BJS recidivism studies are always highest when arrest is used as the outcome.They are always lowest when return to prison for a new crime is the outcome. Intermediate levels of recidivism emerge when new convictions and a return to prison for both new 72

Static Risk Factors and Criminal Recidivism Table 4.3  2014 BJS 3- and 5-Year Recidivism Rates Outcome Rearrest Reconviction Reimprisonment

Number of States

3-Year Recidivism Rate (%)

5-Year Recidivism Rate (%)

30 29 23

67.8 45.2 49.7

76.6 55.4 55.1

Source: Durose et al. (2014, pages 1 and 14).

crimes and violations of supervision conditions are combined. This hierarchy simply reflects the operation of the criminal justice system since there is: (1) case attrition as criminal cases penetrate and sometimes exit the criminal justice system; and (2) the waiting time to an arrest is typically shorter than the waiting time to a new conviction which is itself shorter than the waiting time to a prison commitment for a new crime.9 Consider a couple of examples of this idea in practice. Schmidt and Witte (1989) examined the return-to-prison rates for two separate 1-year North Carolina prison release cohorts (1978 and 1980). Using their split-population lognormal models, they estimated limiting return to prison rates 45% for the 1978 cohort and 49% for the 1980 cohort (page 72). An analysis on which I collaborated (Dean, Brame, and Piquero, 1996) examined rearrest recidivism based on North Carolina training school releasees who were at or over the age of majority in North Carolina at the time of their release (16 years old) also used split-population lognormal models for time to recidivism. We estimated a limiting rearrest failure rate of 77.8% for our sample. Similarly, Table 4.3 presents the recent Durose et al. (2014) 5-year recidivism study of prison releasees in 30 states. A related point is that recidivism is often limited by measurement difficulties and significant measurement errors. Recidivism studies must make clear the definition of recidivism (rearrest, reconviction, reimprisonment, violations of probation and parole supervision conditions) but it is also necessary for researchers to own up to measurement difficulties such as the limitations of criminal record searches. The recent BJS recidivism study described by Durose et al. (2014) shows us what is possible when researchers are able to do cross-state record searches but most state and local area recidivism studies are not able to attain the same scope and depth as this national study (Blumstein and Nakamura, 2012). This is non-random measurement error and it seems likely that recidivism studies generally undercount new criminal justice involvement among people who are highly mobile (particularly those who live in cities near state boundaries). There are also many problems associated with haphazard reporting of arrests for less serious crimes, variation in the reporting of juvenile criminal histories, the widespread use of aliases and efforts to conceal and obscure one’s identity, and the handling of expungements and agreements to drop charges in exchange for successful completion of programs and interventions. All of these issues create the potential for additional measurement error.

Extrapolation Problems A pre-requisite for studying variation in recidivism is the status of having been formally sanctioned for prior offending. A starting point is to define prior sanctions as a “prior police contact” or a “prior arrest.” Christensen’s (1967) work demonstrates that—by this definition—more than half of American males born in the mid-1960’s are at risk for recidivism. My own work looking at self-report survey data from the National Longitudinal Survey of Youth suggests that nearly a third of Americans born in the early 1980’s who were adolescents in the mid-1990’s have accumulated at least one arrest for a nontraffic offense by the time they reach the age of 23 years old (Brame et al., 2012, 2014).This means that a broad cross-section of the U.S. population—not just people who have been 73

Robert Brame

released from prison or people who are on probation—must confront both the direct and collateral consequences of formal criminal justice contact, including the possibility of recidivism. Since the pool of people who potentially could recidivate is so large and heterogeneous, it is useful to think carefully about whose risk of recidivism is being measured and what sanctions or interventions are being imposed on them—before the recidivism clock begins to tick.10 A paper by Manski and Nagin (1998) highlights some of these questions. Their work looks at sanction effects among a cohort of male juveniles in Utah who were born in the early 1970’s and who had a case disposed in the Utah juvenile justice system prior to their 16th birthday. Each of these 13,197 boys was then followed for 24 months to see which ones returned to the juvenile court for a new offense and which ones did not. Manski and Nagin were particularly interested in comparing the recidivism outcomes of boys who received residential placement on the one hand, and community placement on the other. This study did not consider the waiting time to recidivism but instead treated it as a binary outcome (each person was classified as a recidivist or non-recidivist based on whether he appeared in court again within the two-year follow-up period). Manski and Nagin reported that 77% of the boys who received residential treatment returned to the juvenile court compared to a 59% rate among the boys who stayed in the community. This difference is even more striking when one considers that the two-year follow-up period includes the time that the residential placement boys were off the street. A theme of Manski and Nagin’s paper is that the data alone are not strong enough to clearly interpret the policy significance of this difference. The problem is that we do not know why juvenile court judges decided these cases the way they did. To explore this issue, Manski and Nagin (1998:101) propose two theoretical models of judicial decision-making: the skimming model (assign high-risk boys to residential treatment and low-risk boys to community placement) and the outcome optimization model (assign boys to the disposition that minimizes recidivism). It turns out that the data imply a criminogenic effect of residential treatment if judges dispose cases according to the outcome optimization model.The opposite is true (among boys with one or more prior referrals) if judges dispose cases in alignment with the skimming model. Manski and Nagin’s (1998) results highlight a crucial ambiguity: when a prediction exercise results in assignment of a particular treatment or intervention, then our prediction is not only a prediction about one’s future behavior; it also implies a prediction about the causal effect of that treatment on the behavior we are studying (i.e., recidivism). Bushway and Smith (2007) and Rhodes (2011) emphasize the problem of predictions that induce decisions about treatment based on a so-called “statistical treatment rule” when the treatment can be expected to exert causal effects on the behavior being predicted. Once one is assigned to a particular course of treatment based on a statistical treatment rule, we lose the opportunity (except in special circumstances) to observe the counterfactual (i.e., what that person would have done instead if he/she had received a different treatment). Let’s consider an example. A predictive modeling exercise for people who have a particular set of characteristics may be based on recidivism after 18 months of imprisonment followed by 6 months of intensive community supervision. After the predictive modeling exercise is over, we may decide to sentence people with these same characteristics to 24 months of imprisonment followed by 8 months of intensive community supervision. The problem now is that the implemented policy is based on a different sanction regime than what was in place when the risk score was calibrated. In general, we can’t confidently extrapolate our predictions from the first regime to the second regime since a sanction dose that presumably affects recidivism has also changed.11 As Starr (2014:855) points out: The [quantitative prediction] instruments tell us, at best, who has the highest risk of recidivism. They do not tell us whose risk of recidivism will be reduced the most by incarceration. The two questions are not the same, and only the latter directly pertains to the state’s penological interests [emphasis in original; see also the discussion on pp. 859–860]. 74

Static Risk Factors and Criminal Recidivism

So our understanding of apparent predictors of recidivism is compromised when we don’t properly understand the process by which disposition decisions are made or the causal effects of those disposition decisions on the behavior of interest (Rhodes, 2011). It follows that when we study recidivism under a particular set of circumstances and then extrapolate the information from that analysis to new settings, it is necessary to check and see whether the patterns on which the extrapolation was based are still intact. This will inevitably take a period of time to do in any particular application since recidivism studies generally unfold over a period of years (Gottfredson and Moriarty, 2006; Bushway and Smith, 2007; Monahan and Skeem, 2014:162).

Static Risk Factors In this section, I consider the research literature covering several categories of static risk factors, current practice and usage involving those risk factors, and issues needing further study. I begin this section by devoting special attention to the class of characteristics like race and/or ethnicity that people widely object to using as the basis for developing prediction rules. The ideas in this section are also germane for other potentially objectionable background characteristics such as education, neighborhood of residence, and indicators of poverty and/or social class. Next, I turn to the use of other demographic characteristics of individuals such as sex and date of birth that have played significant roles in prediction exercises based on static characteristics. Finally, I consider indicators of past criminal conduct and criminal justice system involvement and issues that might arise as a result of their use in making predictions about future criminal involvement. It is useful to keep in mind that even though much of the literature on static risk factors speaks about their use in actuarial exercises to quantitatively assess recidivism risk, individual decision makers in the criminal justice system (i.e., probation officers, judges, parole boards, etc.) are also able to use these same criteria for making decisions that affect the sanctions, interventions, and services that suspected or convicted offenders receive.

Problematic Predictors of Recidivism Risk An entire class of static factors—those directly linked to race, ethnicity, and religious beliefs—are not implicated in contemporary risk assessment and prediction use in the United States.12 Still, there is some controversy about the way these variables can distort prediction exercises even if they are not explicitly included. It is sufficient for my purposes to consider the issue of race as an example. A question in the literature (see, for example, Harcourt, 2015) is whether race is sufficiently correlated with variables (especially indicators of one’s prior criminal history) that are used in risk assessments that those other variables become de facto “proxies” for race.13 Monahan and Skeem (2015:21) contested this idea arguing that “criminal history is not a proxy for race—instead it overlaps race and possibly mediates race’s relation to recidivism.” Many prediction tools that rely on static risk factors are based on some sort of statistical analysis on a “development” dataset (Starr, 2014:811). The risk assessment instrument then incorporates some sort of rule for accumulating and weighing the factors that contribute useful predictive information about the risk of recidivism. Finally, the risk assessment instrument’s predictions are tested on a “validation” sample to ensure that the instrument performs as expected. A key issue is the role that static predictors like race should play in the initial statistical analysis that informs the deve­ lopment of the instrument. A historically significant example of this type of exercise is the “Salient Factor Score” (Hoffman and Beck, 1974:204), which includes the measures listed in Table 4.3. Other instruments could be used to motivate an identical discussion. The potential problem that arises with any risk assessment instrument is the possibility that certain conditions associated with race (i.e., intensive police patrol in certain neighborhoods, spatial variation in victims’ tendency to call the police, differential patterns of drug enforcement, school-to-prison pipeline, etc.) produce 75

Robert Brame Table 4.4  Salient Factor Score Sheet Items Item Prior Convictions (none, 1-2, > 2) Prior Incarcerations (none, 1-2, > 2) Age at first commitment (< 18 years old, > 18 years old) Commitment offense involved auto theft (yes, no) Previous parole revocation (yes, no) Prior heroin, cocaine, or barbiturate dependence (yes, no) Completed high school (yes, no) Full-time employment or school attendance for at least 6 out of last 24 months (yes, no) Inmate plans to live with spouse and/or children when released (yes, no)

strong covariation between race, recidivism, and some of the factors listed in Table 4.4.14 There are good reasons to be concerned about this problem. Consider, for example, the classic study by Doug Smith (1986), whose examination of police behavior in the neighborhoods of Rochester, St. Louis, and Tampa-St. Petersburg revealed that police enforcement practices (including arrests, use of coercive authority, and filing reports) were stratified by neighborhood socioeconomic status even after controlling for type of crime and other key incident characteristics. I developed a simulation to demonstrate some of the points that have previously been made about this problem in some of the older literature (Fisher and Kadane, 1983; Schmidt and Witte, 1989; Gottfredson and Jarjoura, 1996; Gottfredson and Moriarty, 2006). My simulation has the virtue that we have a good understanding of the process that was used to generate the data (in real datasets the underlying process is usually not very well understood).15 Let’s assume that we have a variable called b that is coded 1 if a person is black and 0 if a person is white. Further, assume we have a variable called c that is coded 1 if the person has any prior felony convictions and 0 otherwise. Finally, let’s say we have an outcome variable called r that is coded 1 if the person is observed to recidivate within some well-defined follow-up period and 0 otherwise. The risk equation of interest is:

∆r ≈ Pr (r = 1|c = 1) − Pr (r = 1|c = 0)



which estimates the difference between the recidivism rates of the people who do and do not have prior records. My simulated population is comprised of 10,000 people and the contingency table of r and c from that population appears in Table 4.5. Based on the information in this table, we would estimate Δr by:



∆r ≈

2199 2369 − ≈ 0.671 − 0.352 ≈ 0.319 1076 + 2199 4356 + 2369

which implies that there is nearly a 32 percentage point difference in the recidivism rates between the two groups. Next, we consider a stratified contingency table (Table 4.6) that takes the variation in b into consideration.The simulation presumes that b is correlated with recidivism because of unmeasured factors that influence recidivism which are correlated with race. Because some of these same unmeasured factors are also presumed to influence whether priors are present, there is a correlation between b and c. Based on the information in Table 4.6, we can calculate two different values of Δr—one for the blacks and one for the whites. These calculations yield (Δr|b = 0) ≈ 0.245 and

76

Static Risk Factors and Criminal Recidivism Table 4.5  Contingency Table of r and c Priors Recidivism

c=0

c=1

Total

r=0 r=1 Total

4,356 2,369 6,725

1,076 2,199 3,275

5,432 4,568 10,000

(Δr|b = 1) ≈ 0.230. While these two Δr values are similar to each other, they are both substantially lower than in the unstratified analysis (Δr ≈ 0.319) because that analysis ignores the confounding effect of race. An average estimate of the two Δr values (after adjusting for race) can be obtained by:

    4,987 5,013 ∆r ≈ 0.237 ≈  × 0.245 +  × 0.230   5,013 + 4,987   5,013 + 4,987 



0.319 − 0.237 ≈ 0.260 .16 So, the bias in ignoring race in this simulated example is on the order of 25%  



0.319

It is useful to recall that Schmidt and Witte (1989:6) argued precisely this point over 25 years ago: In the usual settings in criminal justice, the available information generally consists of extensive data on the attributes, experiences, and activities of the individuals. The set of such information for prediction generally involves two steps. First, a set of individual data is used to estimate a model. Second the model is used, together with information on an individual, to predict the individual’s future behavior. In our opinion, ethical questions about the type of information that should be used arise only in the second step of this process. However, it is important to emphasize that the only way to be sure that an unacceptable piece of information (e.g., race) is not included in the second step of this process is to include it in the first step [emphasis in original]. Of course, the question then arises of how to properly adjust the second step to account for the differences between the analysis and the actual prediction exercise (Fisher and Kadane, 1983). Schmidt and Witte’s treatment of the issue along with the work of Gottfredson and Jarjoura (1996) offers some helpful considerations and recommendations for criminologists in this situation.17

Table 4.6  Contingency Table of r and c Stratifying by b b=0

b=1

Priors

Priors

Recidivism

c=0

c=1

Total

c=0

c=1

Total

r=0 r=1

3,091 1,161

367 394

3,458 1,555

1,265 1,208

709 1,805

1,974 3,013

Total

4,252

761

5,013

2,473

2,514

4,987

77

Robert Brame

Age and Sex The demographic characteristics of age (anchored to the static date of birth) and sex are among the most well-documented correlates of criminal behavior (see e.g., Blumstein et al., 1986; Blumstein and Cohen, 1987; Gottfredson and Hirschi, 1990). There is controversy about their use in predictive exercises because—like race, ethnicity, the neighborhood in which one lives, family background, etc.—they are extralegal characteristics.18 Debate notwithstanding, it is not unusual to find these factors included in a wide range of risk assessment activities. Starr’s (2014:805) critique of such predictive efforts documented the diffusion: These jurisdictions are directing sentencing judges to explicitly consider a variety of variables that often include socioeconomic status, gender, age, family, and neighborhood characteristics—not just in special contexts in which one of those variables might be particularly relevant (for instance, ability to pay in cases involving fines), but routinely, in all cases. This is not a fringe development. Courts in at least twenty states are already implementing some form of it. One state supreme court has already enthusiastically endorsed it. And it now has been embraced by the American Law Institute in the draft of the newly revised Model Penal Code—a development that reflects its mainstream acceptance and may soon augur much more widespread adoption. Naturally, there are legitimate concerns that the goal of a prediction exercise is to predict well and one might argue that variables like age and sex support that effort. However, a prominent and highly-cited meta-analysis conducted by Paul Gendreau and his colleagues (1996) suggested that neither age nor sex is among the strongest predictors of recidivism (see related discussion in Oleson, 2011:1361–2 and 1365–6).19 Still, because most recidivism studies have been able to measure predictive effects of these variables, it is not hard to understand the motivation for including them in prediction instruments.Various legal theories to support or refute the use of these characteristics for risk prediction are a topic of much contemporary discussion. For detailed consideration of both the supporting rationales and objections that have arisen recently (as more states have been exploring and adopting predictive recidivism risk assessments), I refer the reader to Monahan and Skeem (2014, 2015),Tonry (2014), Starr (2014, 2015), Hannah-Moffat (2013), Kern and Bergstrom (2013), and Oleson (2011). It is worth pointing out that most of the issues considered in this literature are not new (see, for example, Harcourt, 2006; Farrington and Tarling, 1985; Blumstein et al., 1985; Greenwood, 1982; Morris and Miller, 1985;Tonry, 1987; Petersilia and Turner, 1987; Schmidt and Witte, 1989:138–149).

Criminal History Variables In a detailed Appendix, Oleson (2011:1399–1402) provides a comprehensive listing of the measures included in both historical and contemporary recidivism risk assessment instruments. Every single one of the instruments listed contains a significant presence of criminal history measures. It seems that the practice of emphasizing criminal history factors in risk assessment is grounded in three core ideas: (1) past criminal justice system involvement is the best known predictor of recidivism; (2) it is and always has been legally permissible to consider past criminal behavior as a marker for future risk; and (3) the errors, problems, and shortcomings of criminal history data that are likely to be available for risk assessment are outweighed by the predictive value offered by whatever measures are available. The terrain of decisions about which information from criminal history can be brought to bear is growing wider:

78

Static Risk Factors and Criminal Recidivism

Notably, offense record considerations in recent years have influenced decisions in such criminal law areas as the availability of bail, probation conditions, parole release, security level determinations, intensity of programming, and requirements for criminal registries. (Hamilton, 2015:80) Despite the ubiquity with which criminal history variables appear in risk assessment instruments, there are normative debates about whether this is proper. Hamilton (2015) provides a comprehensive review of the legal and philosophical debates surrounding the use of prior criminal history information in the criminal justice system. The main issues in contention seem to be: (1) whether it is fair to sentence one more harshly at one point in time because of criminal behavior and sanctions administered at some previous time point (a question proposed by some retributive theories); and (2) whether there are important social benefits for the greater good when we pay attention to and place weight on evidence of incorrigibility in our criminal justice response to people who violate the law (a question proposed by some utilitarian theories). Of course, the American criminal justice system has always recognized both retributive and instrumental goals; as a practical matter this means that prior criminal history has traditionally been and continues to be an influential factor with judges, parole boards, and other criminal justice decision makers. Risk assessment instruments have the potential to institutionalize and standardize the role that such information contributes to criminal justice decision making. With this context in mind, I now turn to some more specific issues related to the applied use of criminal history information in recidivism risk assessment. The most straightforward measure of prior criminal history might be some indicator of whether someone has ever before been convicted of a crime. This is a yes or no question but it raises some surprisingly complicated issues. The recent Bureau of Justice Statistics recidivism study (Durose et al., 2014:16–18) offers a detailed description of the processes that had to be followed to obtain both pre- and post-release criminal histories on the prison releasees in the 30 states included in the study. It is inconceivable that local and state jurisdictions conducting their own searches for criminal history information would be able to duplicate the same extensive process used by the BJS research team. Depending on the quality of the state’s own criminal history repository and the ability to do a comprehensive national search, the quality of a jurisdiction’s criminal history information will vary from place to place (and possibly over time as well as demonstrated by a recent survey of state criminal justice information systems from the Bureau of Justice Statistics (2014)).20 In any event, the BJS study tells us that the median number of prior convictions for the released inmates is 3.1 but it does not say how many released inmates had at least one prior conviction. We know it has to be more than 50% but we don’t know how much.21 If one were to build a model to see how well a prior conviction predicted recidivism, the first obstacle to be cleared is actually getting accurate information. Similar issues would arise with arrest, police contacts, or other measures of criminal justice involvement. Let’s suppose one were able to actually procure reasonably accurate information about prior conviction experiences. The next issue is how that information should be used in a statistical model. Perhaps the presence/absence of a conviction at age 20 means something different than it does at age 40. Most of the risk assessment instruments that I reviewed in the course of writing this paper assigned or subtracted points based on age and criminal history separately but not in an interactive way.22 This would seem to be an area where further research is needed (see, for example, Dean et al., 1996).

79

Robert Brame

For some years now, I have been interested in the question of how much predictive weight should be assigned to prior criminal records (like a prior arrest or conviction) when those events occurred long ago in comparison to criminal history events that occurred more recently (Kurlychek et al., 2006; 2007; see also Blumstein and Nakamura, 2009, 2012). Recidivism studies are virtually unanimous in their demonstration of a declining hazard for recidivism as time since the last criminal justice event increases (as discussed in Section 2.5 above). This matters because of an insight from Bushway et al. (2001) which suggested that desistance from crime may be a process rather than an event. If the evidence suggests that someone’s waiting time to the next criminal justice system encounter is getting longer, that would be compatible with the idea that a person who has offended in the past is “slowing down” their trajectory of offending. Despite the potential importance of this factor, it doesn’t seem to be explicitly considered in most of the risk assessment instruments that have been used in the past or that are being used today.23 Another controversial issue in this literature revolves around the use of juvenile criminal history information and arrest information in assessing recidivism risk. Several of the instruments described in the Oleson (2011) appendix refer explicitly to variables in both of these categories. Consider the issue surrounding the question of counting arrests as a matter of risk assessment: An initial and overarching issue is that such an assessment violates the espoused tenet of western criminal law systems that a person is assumed innocent until proven guilty. As a result, there is a strong argument that evidence of criminality outside of convictions ought not to be relied upon in legal decisions, particularly those that result in significant infringements upon liberty and privacy. Nevertheless, risk instruments generally permit coding for criminal history measures without requiring convictions. Hence, the potential for weak, if not entirely inaccurate, information to guide risk assessment outcomes is real. Add to this vulnerability the prospect that the alleged prior offending may simply replicate discriminatory practices already existing in criminal investigation processes. (Hamilton, 2015:105) There are similar sorts of issues around the use of juvenile records: (1) there is great jurisdictional variability in the treatment of juvenile criminal records; (2) an adjudication of “delinquent” in juvenile court is not recognized as equivalent to an adult conviction in our legal system; and (3) the fairness of developmental issues identified by the Supreme Court in several recent cases having consequences that propagate into adulthood (Hamilton, 2014:112; In re Gault, 387 U.S. 1, 1967; Roper v. Simmons, 543 U.S. 551, 2005). Some instruments such as the LSI-R actually do both of these things at the same time, relying explicitly on the number of arrests one experienced before age 16 (Hamilton, 2015:94). In my own research (Kurlychek et al., 2006, 2007), we discovered that even though juvenile criminal justice contact was predictive of adult contact, the predictive value of that information decayed rapidly and dramatically over time as people move into adulthood. Despite these concerns, the use of both arrest and juvenile criminal history information seems to be a prominent feature of the contemporary risk assessment landscape.

Conclusions Based on the foregoing overview of the literature on static risk factors and criminal recidivism, I see a few key ideas worth emphasizing. First, while it is established that most arrests involve people who have been arrested before it does not follow that criminal recidivism is the primary driver of American crime rates, particularly serious or index crime rates (Gottschalk,

80

Static Risk Factors and Criminal Recidivism

2015) although as Rosenfeld, Wallman, and Fornango (2002) have pointed out, the contribution of recidivism to overall crime patterns can and does vary over time. We need to keep the basic distinction between crimes and arrests in mind, recognizing that arrests reflect victims’ decisions to report crimes to the police and the behavior of the police themselves—in addition to the behavior of people who are arrested. This also holds true when we consider the population of people who formally encounter the criminal justice system (i.e., people who have ever been arrested). Broadly speaking, this is the group who is at risk to recidivate. Criminal activity coupled with victim and police actions is the gateway into the system—not just criminal activity alone. Second, as both Blumstein and Moitra (1980) and Schmidt and Witte (1989) have noted, we need to bear in mind that criminal recidivism is at least partially a probabilistic process. Some people at risk to recidivate commit new crimes while others don’t. Some of those who do commit more crimes are apprehended by the police (or revoked for technical violations of supervision conditions). Some of those who are apprehended penetrate further into the criminal justice system while others don’t. Each of these stages has systematic and stochastic elements. Those elements are likely to vary from person to person and from place to place. We can make some reasonable adjustments for factors such as neighborhood and community of residence, the proper modeling of decisions at later stages of the criminal justice system conditioning on decisions that were made at earlier stages of the system, and the development of better prediction algorithms that abandon the rigidity of linear and additive regression-based approaches to risk assessment in favor of less parametric and more flexible approaches. Current developments in the area of machine learning would seem to be the cutting edge of this movement (Berk and Bleich, 2013).24 So, we certainly need more research and interdisciplinary engagement with emerging methods of prediction and how predictions are transformed into decisions about what will happen to specific people. Efforts to improve our practice notwithstanding, it is quite plausible that the intrinsic randomness of individual behaviors, environments, and experiences will create severe upper limits on the amount of recidivism that can ever be explained or predicted with even the most appropriate data and statistical modeling tools (i.e., machine learning methods). Third, when one conducts a recidivism study there is generally an opportunity to measure not only whether a recidivism event occurs but also the timing of its occurrence. Both Maltz (1984) and Schmidt and Witte (1989) have argued persuasively that systematic study of the timing of recidivism is fundamentally part of developing a good understanding of the process by which people return to offending behavior. When we do this, it is possible to more fully comprehend and understand how recidivism is unfolding over time. In short, some people fail quickly while others take longer. It seems important to make distinctions between these two groups of people. Binary recidivism measures, therefore, do not efficiently use all of the information that is generally available in the data (Maltz, 1984; Schmidt and Witte, 1989). Incorporating waiting times to recidivism—not just the presence or absence of recidivism—in our prediction efforts is a high priority for future research. One of the useful ideas from the literature on criminal careers is that the processes that produce variation in different dimensions of criminal offending may vary considerably. Toward that end, researchers like Schmidt and Witte (1989) have explored whether the factors associated with whether someone recidivates are the same as those that predict the timing of recidivism. Although Schmidt and Witte’s work was conducted over 25 years ago, we still don’t have enough research like this. Similarly, we need more exploration of whether the predictors of, say, serious violent recidivism are the same as the predictors of recidivism more generally. Another area in need of further research is the extrapolation problems outlined in ­Section 2.8. When we contemplate policy changes based on the results of a recidivism study,

81

Robert Brame

a high priority needs to be placed on understanding whether the recidivism patterns that motivated the policy change have themselves shifted afterward (Bushway and Smith, 2007; Monahan and Skeem, 2014). Relatedly, when a judge, a probation officer, a parole board or a quantitative tool makes a prediction about someone’s future behavior, a prediction is also being made about the effect of interventions, sanctions, and services that will be administered as a result of that prediction (Rhodes, 2011; Starr, 2014). Unfortunately, well-designed efforts to confirm the robustness of pre-change patterns will themselves take long periods of time to conduct. Bushway and Smith (2007) provide several compelling examples of how shifts in the patterns of behavior can occur after policy changes which were based on pre-existing behavior patterns have been implemented. It turns out that the problem of eliminating concerns about racial bias is more complicated than simply ignoring race in predictive models. In fact, one of the important lessons of the methodological literature on risk assessment is that leaving variables like race and ethnicity out of our recidivism risk assessments guarantees that they will still be there.This is an idea that has been understood for many years but as Gottfredson and Moriarty (2006) point out, it continues to exist. In my view, the field needs to devote more effort to thinking carefully about the role of race and other static extralegal characteristics in criminal justice decision making (regardless of whether those decisions are guided by formal risk assessment or pure discretion). There are a number of concerns around the use of static criminal history variables that require more attention. A principal issue is how criminal history interacts with age. If two people have the same scores on prior criminal history variables but those two people are different ages, the predictive effects of those variables on recidivism may very well not be additive. But it appears that most risk assessment scales treat them as additive. A second issue has to do with the uneven coverage of criminal history information across jurisdictions. One approach to this problem is to say that it doesn’t matter. Some criminal history measures are better than nothing at all. But the point remains that criminal history can influence key dispositions and people stand to be penalized based simply on the quality of a jurisdiction’s record keeping systems. It also appears that most risk assessment instruments take little or no account of the amount of time that has lapsed since one’s last contact with the criminal justice system. We need more research to see whether the “gliding to desistance” hypothesis offered by Bushway et al. (2001) can inform the use of risk assessment measures that recognize the reduced recidivism risk of people whose criminal careers appear to be slowing down. Finally, the field also needs more thought about the propriety and wisdom of using juvenile criminal history information and information about arrests, particularly arrests that don’t turn into convictions (Stevens and Morash, 2014:7) in its assessment of recidivism risk. Like the other issues considered in this essay, this endeavor requires careful consideration of what exactly it is we are trying to do when we predict the risk of recidivism and what decision makers should do with that information. Then, there are concerns about which measures we use to make predictions and the technical details of how the measures are transformed into scoring instruments that are used to make predictions about the future behavior of individual people. It involves attention to both the limits of our theory and methods and concern about what is fair and just; and, it is a shared responsibility between social scientists, criminal justice practitioners, and the legal community.

Notes 1 University of South Carolina, March 2016. 2 I do not address the literature on dynamic risk factors, which consider person-specific features that can change and evolve over time such as addiction, accessing treatment and services, employment, housing, peer networks, and medical care. An excellent recent survey of the issues concerning the coexistence of

82

Static Risk Factors and Criminal Recidivism static and dynamic risk factors for criminal recidivism can be found in Caudy, Durso, and Taxman (2013). Their study reports that dynamic risk factors do not significantly improve recidivism risk predictions over and above predictions based on static factors alone. 3 The FBI’s arrest reporting program has historically experienced lower participation than the crime reporting program. For example, in Christensen’s base year of 1965, the U.S. population was estimated by the U.S. Census to be about 194.4 million persons while the population covered by the arrest reporting program was about 134.1 million persons, or a 69.0% coverage rate. For the year 2014 (the most recent year available), 12,327 law enforcement agencies voluntarily contributed age-graded arrest data covering approximately 250 million people (about 78% of the nation’s estimated population of 319 million people that year). Even though the coverage of the arrest reporting program is significantly lower than the coverage of the more widely publicized crime reporting program, it is probably the best information we have on arrest frequencies by age in the United States. Christensen used some relatively simple adjustments to scale up the arrest frequencies to cover the entire United States. 4 While this sounds like a dramatic level of skew, Blumstein (2010:552) reminds us that when we confine the analysis to people who were contacted at least once, about 18% of those people were responsible for about 52% of the contacts—a finding he considers “far less provocative.”. 5 In addition, when we study person-to-person variation in counts of criminal justice system contacts, arrests, convictions, or violations of supervision conditions we must be mindful that this variation can also be partially explained by variation in patrol, enforcement, and supervision practices of criminal justice system actors (see, for example, Blumstein et al., 1986). 6 An example of this appears in Tables 9 and 10 of Durose et al., (2014:9). Table 9 shows that the modal rearrest category for persons released from prisons in 30 states is “public order offenses.” This pattern is true regardless of whether the offender served prison time for a violent offense (55.3% were rearrested for public order offenses), a property offense (61.9% were rearrested for public order offenses), a drug offense (56.1% were rearrested for public order offenses), or a public order offense (59.6% were rearrested for public order offenses). 7 The code to reproduce this analysis is presented in Appendix 1. 8 Blumstein and Graddy (1982) found that race differences in arrest rates were more clearly attributable to differences in participation, not differences in recidivism among those who participated. 9 Which definition is best to use is not a settled matter. Of course, a preference for one measure over another must depend on the research or evaluation question to be answered. Institutional corrections officials will likely be most interested in a measure of recidivism that emphasizes how often people return to prison (Maltz, 1984; Schmidt and Witte, 1989). On the other hand, if we are trying to reach a better understanding of whether people have desisted from offending, a measure based on future arrests or convictions may be more appropriate. Arrests (and arrest charges) involve less filtering by different stages of processing in the criminal justice system so they are potentially a more accurate reflection of actual behavior; convictions have the advantage that a person has actually been found guilty (or, more often, pled guilty) to an offense so arrests based on weak evidence would be disproportionately filtered out. The most detailed and even-handed discussion of this issue I have seen is Maltz (1984:54–67). 10 Some research projects investigate the effects of various experiences, sanctions, services, surveillance and treatment strategies, and interventions after the recidivism clock begins to tick. These studies of so-called “dynamic factors” present special issues that are beyond the scope of this paper. 11 A counterargument to this proposition in the case of a judge sentencing someone to prison or a parole board keeping someone in prison is that we can be certain of the incapacitative effect of that decision. There would seem to be (at least) two responses to this counterargument: (1) the counterfactual is still unobserved (i.e., we don’t know what that person’s behavior would be if he/she was released); and (2) the vast majority of people in prison do return to the community at some point so we are not released from the responsibility to think carefully about the effects of incarceration on a person’s post-release behavior, current incapacitation benefits notwithstanding. 12 Tonry (2014) summarizes the issue: “Race, ethnicity, and religion are not to my knowledge anywhere used as an explicit factor in prediction instruments in sentencing or parole policies. However, the use of any of them likely would be upheld, as it was in the profiling cases, so long as it was only one among several factors. Explicit use of race, ethnicity, or religion, however, is widely regarded as unseemly, and so the issue is unlikely to arise.” Oleson (2011:1356–1357) also reaches this conclusion. 13 Other terminology for this phenomenon might be discrimination in the operation of a rule (as opposed to discrimination in the application of a rule}) (Kaye, 1982) or the idea of disparate impact (as opposed to disparate treatment) (Starr, 2014:803; Griggs et al. v. Duke Power Company, 401 U.S. 424, 1971).

83

Robert Brame 14 Such variables might implicate other extralegal static or nearly-static factors like the neighborhood in which one lives and indicators of one’s social class (education, job stability) or family characteristics. 15 The code to reproduce this simulation is presented in Appendix 2. 16 If the values of Δr differ widely across race groups, then we would conclude that race moderates the predictive value of priors and we might want to maintain separate estimates. 17 There are examples of this insight in the contemporary extant literature: two risk assessment ­instruments— one that is currently being developed in Pennsylvania and one developed in 2002 in Virginia—report ­controlling for race as in the first step identified by Schmidt and Witte (1989) but exclude race in the second step (Ostrom et al., 2002:28, footnote #10; Pennsylvania Commission on Sentencing, 2015:11, footnote #5). 18 One prominent response to this objection is that extralegal characteristics have always been part of discretionary decision-making in the criminal justice system. Proponents of this view (see, for example, Monahan and Skeem [2014, 2015]), typically allow that legal characteristics should be of primary importance in establishing the broad parameters of sentences, sanctions, and interventions. But within these parameters, formal risk assessments can and should be used to promote rigor and consistency in the handling of extralegal characteristics that are meaningfully associated with recidivism risk. The opponents concede no ground. For example, Starr (2014) contends that when formal risk assessment instruments are presented to decision-makers, the priority assigned to the factors in those instruments is vaulted to a higher level than would otherwise be the case. 19 This echoes a key finding from Blumstein and Graddy (1982) that race is predictive of initial formal contact with the system but conditional on that initial involvement the relationship between race and recidivism is much weaker. 20 One of the principal difficulties in criminal history searches is considering events that occurred outside the state in which the jurisdiction lies. For example, the BJS recidivism study showed that almost 25% of the released inmates had some sort of prior arrest outside the state where they served time in prison. 21 Unfortunately, it appears we will not be able to know what this estimate is unless the BJS research team produces it. When I inquired to the BJS about the availability of any of the data for research purposes, I was informed that there are currently no plans for independent non-BJS researchers to be permitted to study or access the data. 22 An interesting emerging exception to this pattern is the risk assessments being conducted by Berk and Bleich (2013) who have been using machine learning methods that allow for precisely these kinds of interaction effects. 23 For a similar set of ideas, see Scott, Foss, and Dennis (2005) who illustrated how the length of spells between treatment and relapse can be a useful indicator of progress in treatment. 24 Bushway (2013) provided a useful commentary on the question of how well standard regression models compare with the newer machine learning approaches. He concluded that the field has so far largely not engaged with these new methods and that the field will benefit from greater attention and focus on both their capabilities and limitations:“I recognize that the field of criminology has mixed feelings about the use of risk-prediction tools. However, these mixed feelings are largely absent from the world of criminal justice, where practitioners are under pressure, often from lawsuits, to be more effective with fewer resources. The use of risk-prediction tools is already substantial, and it is growing rapidly (Harcourt, 2007; Simon, 2005). Criminology as a field can choose to ignore this, or it can help build the science and practice of risk prediction’’ Bushway (2013:566).

Appendix 1 Kitchener Recidivism Data This Appendix presents the R code used to generate charts based on the Kitchener et al. (1977) federal prisoner recidivism data described in Sections 2.3 and 2.5. # convert data to machine readable form # first, use the aggregate data year