Tools for Decision Making: A Practical Guide for Local Government [3 ed.] 0367654393, 9780367654399

Citation preview

“Finally, a no-nonsense, how-to book that takes the fear out of statistics and analysis. Tools for Decision Making is an ideal book to use in masters-level research methods classes where the emphasis is on equipping students with basic tools of analysis that they can use in their careers.” — David Schultz, Distinguished University Professor of Political Science and Legal Studies at Hamline University, USA “By incorporating Excel instructions into the collection of analytic techniques presented in Tools for Decision Making, the authors have made this book a must-have. Every local government researcher/analyst in the world has access to Excel, so this feature alone makes this text invaluable. The online Excel templates that accompany the third edition are a major selling point; their availability for student use and for subsequent use on the job make this book a necessity on one’s bookshelf.” — Hunter Bacot, Graduate Program Director and Professor of Political Science at the University of North Carolina at Greensboro, USA “Tools for Decision Making is a gold mine of ideas for improving services. It makes difficult-sounding management techniques very understandable. I caution owners of the book to be leery of those who seek to borrow it; else it may go the same way as the trusty hammer and saw.” — W. Bartley Hildreth, Professor at the Center for State and Local Finance / Department of Public Management and Policy at Georgia State University, USA “The analytical tools described by Ammons and Roenigk demystify the management of complex organizations. This book can serve as your tool box of ideas and models for continuous improvement.” — Russell Blackburn, City Manager at the City of Port St. Lucie, Florida, USA “Tools for Decision Making is a very user-friendly book that can be used in policy analysis or financial management courses in upper-division or graduate instruction. Ammons and Roenigk use excellent examples that place tools in realistic and appropriate context for typical public administration students.” — Howard Frank, Professor in the Department of Public Policy & Administration at Florida International University, USA

TOOLS FOR DECISION MAKING

This book guides readers to the mastery of a wide array of practical analytic techniques useful to local governments. Written in an easy-​to-​read style with an emphasis on providing practical assistance to students, local government practitioners, and others interested in local government performance, this updated third edition features analytic methods selected for their relevance to everyday problems encountered in city and county governments. The authors outline a variety of practical techniques including the simplest that the fields of management, public administration, policy analysis, and industrial engineering have to offer. Each analytic technique is introduced in the context of a fictitious case presented over a few pages at the beginning of that technique’s chapter. Contents include demand analysis, work distribution analysis, process flow-​charting, inflation adjustments, annualizing capital costs, staffing analysis, identifying full costs of a program or service, present value analysis, life-​cycle costing, lease/​buy analysis, cost-​ effectiveness analysis, benchmarking analysis, and more. This updated third edition features a dramatic expansion of Excel-​based applications, plus templates and exercises accompanying many of the chapters and available online. New chapters prepare readers to: • • • • • •

use statistical tests to identify significant differences in performance averages; construct Pareto charts; develop cause-​and-​effect diagrams; prepare control charts; detect possible discrimination in hiring and appointment practices; and present analytic evidence more effectively.

This book is an essential resource for students and instructors of public administration courses on analysis, methods, evaluation, productivity improvement, and service delivery. Online resources for this book, including Excel templates, are available at https://​toolsfordecisionmaking.sog.unc.edu David N. Ammons is a Professor of Public Administration and Government at the University of North Carolina at Chapel Hill, USA. He has written several books and his articles have appeared in Public Administration Review, Journal of Public Administration Research and Theory, Public Performance and Management Review, and other public affairs journals. He has served on the National Performance Management Advisory Commission and the North Carolina Governor’s Advisory Committee on Performance Management. He was elected as a Fellow of the National Academy of Public Administration in 2006. Dale J. Roenigk is the Director of the North Carolina Benchmarking Project and a Faculty Member at the School of Government at the University of North Carolina at Chapel Hill, USA. He is the author of an annual benchmarking report on municipal services published by the School of Government. He also helped develop the County and Municipal Fiscal Analysis tool, a web-​ based dashboard designed to help North Carolina local governments analyze their fiscal condition. His research interests include benchmarking, performance measurement and management, and evaluation.

TOOLS FOR DECISION MAKING

A Practical Guide for Local Government Third Edition

David N. Ammons and Dale J. Roenigk

Third edition published 2022 by Routledge 2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN and by Routledge 605 Third Avenue, New York, NY 10158 Routledge is an imprint of the Taylor & Francis Group, an informa business © 2022 David N. Ammons and Dale J. Roenigk The right of David N. Ammons and Dale J. Roenigk to be identified as authors of this work has been asserted by them in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. First edition published by CQ Press 2002 Second edition published by CQ Press 2009 British Library Cataloguing-​in-​Publication Data A catalogue record for this book is available from the British Library Library of Congress Cataloging-​in-​Publication Data Names: Ammons, David N., author. | Roenigk, Dale J., author. Title: Tools for decision making: a practical guide for local government / David N. Ammons and Dale J. Roenigk. Description: Third edition. | Abingdon, Oxon; New York, NY: Routledge, 2022. | Includes bibliographical references and index. Identifiers: LCCN 2021005408 (print) | LCCN 2021005409 (ebook) | ISBN 9780367654399 (hardback) | ISBN 9780367654320 (paperback) | ISBN 9781003129431 (ebook) Subjects: LCSH: Local government–Decision making. | Public administration–Decision making. Classification: LCC JS78 .A49 2022 (print) | LCC JS78 (ebook) | DDC 352.3/7214–dc23 LC record available at https://lccn.loc.gov/2021005408 LC ebook record available at https://lccn.loc.gov/2021005409 ISBN: 978-​0-​367-​65439-​9 (hbk) ISBN: 978-​0-​367-​65432-​0 (pbk) ISBN: 978-​1-​003-​12943-​1 (ebk) Typeset in Joanna by Newgen Publishing UK

CONTENTS

List of boxes List of figures List of tables List of worksheets Preface

xi xiv xviii xxii xxiii

Part I  Introduction

1

1 The role of analysis in local government

3

Part II  The basics

25

2 Central tendency and dispersion

27

3 Performance measurement and monitoring

38

4 Smoothing data trends by using moving averages

60

5 Sampling for analysis

69

6 Basic statistical testing: Correlation and chi-​square

80

7 Sensitivity analysis

99

viii

cont e nts

8 Break-​even analysis

107

9 Detecting meaningful differences in average performance

112

Part III  A  nalyzing a unit’s capacity to meet the demand for service

125

10 Demand analysis

127

11 Work distribution analysis

139

12 Using performance standards

149

13 Staffing analysis

163

14 Staffing factor calculation: Projections for uninterruptible services

180

Part IV  What does it cost?

187

15 Adjusting for inflation when comparing revenues or expenditures

189

16 Basic investment calculations: Figuring interest rates and yields

201

17 The time value of money: Opportunity costs, discounting, compounding, future value, and present value

214

18 Simple options for annualizing costs of capital items: Usage-​rate and straight-​line depreciation

224

19 Identifying full costs of a program

236

20 Calculating go-​away costs for privatization decisions

257

21 Life-​cycle costing

263

conte nts

22 Lease or buy?

272

23 Cost-​effectiveness analysis: A truncated version of cost-​benefit analysis

283

Part V  Process improvement

297

24 Pareto charts

299

25 Cause-​and-​effect diagrams

308

26 Process flow charts

319

27 Control charts

333

Part VI  Other analytic opportunities and techniques

357

28 Citizen surveys and customer surveys

359

29 Analyzing survey data: Revealing graphics and simple statistics

369

30 Financial condition analysis

384

31 Random chance or bias? A practical use of binomial probability distribution

400

32 Forecasting

416

33 Analysis of operations via benchmarking

434

Part VII  Presenting analytic evidence

453

34 Presenting analytic evidence: Tables or graphs?

455

Part VIII  Wrap up

471

35 The place of analysis in a political environment

473

ix

x

cont e nts

Appendix A. Distribution of chi-​square Appendix B. Distribution of t Appendix C. Future value interest factors Appendix D. Sum of annuity factor (SAF) Appendix E. The normal distribution and table of z-​score probabilities Appendix F. Excel’s data analysis ToolPak: Installation, use, and example Index

476 478 480 483 486 490 497

LIST OF BOXES

From the electronic toolkit 1.1 Entering Data in a Spreadsheet 1.2 Creating Line Charts and Bar Charts 2.1 Finding the Mean, Median, and Mode 2.3 Finding the Standard Deviation and the Interquartile Range 4.1 Moving Averages 5.1 Drawing Random Samples by Spreadsheet 5.2 Creating a Random Numbers List from Websites 6.1 Finding the Correlation Coefficient 7.1 “What If Analysis” in Excel Spreadsheets 11.1 Creating Work-​Distribution Charts 14.1 Calculating the Staffing Factor 16.1 Interest Calculations 16.2 Calculating Payment Amounts for a Loan 17.1 Finding Present Value and the Discount Factor 18.1 Finding Usage-​Rate Allocation of Cost 18.2 Straight-​Line Depreciation 19.2 Making Cost Worksheets 21.1 Calculating Life-​Cycle Costs 22.1 Finding the Uniform Series Capital Recovery Factor 29.1 Testing for Statistical Significance

11 13 30 34 65 71 72 84 104 144 182 206 208 220 228 229 242 268 276 377

xii

list of boxes

29.2 On-​Line Chi-​Square Calculators 31.2 Calculations for Using the Binomial Probability Distribution with Excel 32.2 Trend Analysis Avoid this common error 2.2 Beware of Averaging the Averages 5.3 “Random” Does Not Mean Haphazard 6.2 Correlation Is Not Causation 12.1 Beware of Self-​Serving Input Standards 13.2 Basing Staffing Analyses on Employees-​to-​Population Ratios 15.2 Accounting for Changes in Inflation or Population Change by Addition or Subtraction 16.3 Assuming a 20% Gain Followed by a 20% Loss… 28.1 Assuming That a Response Rate of 20 Percent Is Good Enough 29.3 “Percentage” or “Percentage Points”? 29.4 Cavalierly Comparing Scores across Local Government Functions 32.1 Misinterpreting a Straight, Ascending Trend Line as Depicting a Constant Rate of Growth Other 2.4 Other Applications of Measures of Central Tendency and Measures of Dispersion 3.1 Performance Measures Featured in Charts or Dashboards 3.2 Logic Models 3.3 Evaluating Service Effectiveness from Performance Records: A Fire Inspection Example 3.4 Building Performance Measures into Management Systems 3.5 Other Applications of Performance Measurement 4.2 Other Applications of Moving Averages 5.4 Other Applications of Sampling 6.3 Other Applications of the Correlation Statistic 9.1 When Are a Study’s Findings Significant? 10.1 Other Applications of Demand Analysis 11.2 Other Applications of Work-​Distribution Analysis 12.2 Other Applications of Performance Standards

378 404 421 31 74 88 152 170 196 210 361 379 381 419

36 39 46 48 57 58 67 78 90 120 137 147 157

list of box e s

12.3 13.1 13.3 14.2 15.1 15.3 16.4 17.2 18.3 18.4 19.1 19.3 19.4 20.1 21.2 22.2 23.1 24.1 25.1 26.1 27.1 27.2 27.3 28.2 29.5 31.1 31.3 31.4 33.1 33.2 33.3

Use of Different Kinds of Standards Achieving a Full Workload: What’s Optimum? Time & Motion and Time Allocation Studies Other Applications of Staffing Factor Calculations Access to Inflation Indexes Other Applications of Inflation Adjustments Basis Points Other Applications of Present Value Analysis Taking a Second Look: Sensitivity Analysis Other Applications of Depreciation Calculations Activity-​Based Costing Uniform Cost Accounting for Comparative Performance Measurement Projects Other Applications of Cost Accounting Other Applications of Go-​Away Costs Analysis Other Applications of Life-​Cycle Costing Other Applications of Lease-​Buy Analysis Analyzing the Differing Benefits of Equal-​Cost Alternatives Pad-​and-​Pencil Exploratory Pareto Analysis Post-​It Notes and Cause-​and-​Effect Diagrams Another Application of Process Flow Charts Is a Control Chart Really about Control? Why Set the Upper and Lower Control Limits at Three Sigma? A Few Tips When Working with Control Charts Focus Groups Other Applications of the Chi-​Square Statistic Calculations for Using the Binomial Probability Distribution When Are a Study’s Findings Significant? A Word of Caution When Combining Groups for Analysis Scatterplot Patterns Key Statistics Related to Regression Other Applications of Benchmarking

160 165 174 186 195 199 212 222 231 234 239 253 254 262 270 281 285 304 313 328 334 342 351 363 382 403 410 413 442 444 446

xiii

LIST OF FIGURES

2.1 3.1 3.2 3.3 3.4 3.5 3.6 3.7

Range and Interquartile Range: Box-​and-​Whiskers Plot Emergency Turnout Time in Pinehurst Building Inspection Efficiency in Pinehurst Fleet Availability in Pinehurst Logic Model Crew Performance Chart Average Production Units of District Crews Average Labor Hours Required to Repair a Water Main Break 3.8 The Performance Management Cycle 4.1 Using the Five-​Day Moving Average to Monitor New Coronavirus Cases in the US 4.2 Using the Seven-​Day Moving Average to Monitor New Coronavirus Cases in Washington State 5.1 Meter Read Testing Process and Result 6.1 Scatterplot Depicting the Relationship between Miles Driven and the Cost Per Mile for Owning and Operating Light-​Duty Vehicles in King County, Washington 6.2 San Diego’s Fire and Rescue Incidents by Time of Day for Three Recent Years 9.1 The Means Indicate a Performance Difference, But Dispersion Patterns Suggest It Is Not Very Big or Especially Praiseworthy

34 40 41 41 46 53 55 56 57 63 64 78 81 91 113

list of figure s

9.2 Dispersion Patterns Clarify the Superior Performance of Technician 2 10.1 Average Number of Calls for Police Service, by Hour of the Day 10.2 Plan for Revised Deployment of Newbern’s Patrol Officers 10.3 Demand Profile for Business Licenses and Permits 12.1 Armadillo City Garage Equipment Work Order 12.2 Austin Fire Department (AFD) Performance Compared to NFPA Standards 12.3 Oakland’s 911 Call Answering Performance Compared to the State Standard 12.4 Austin’s Drinking Water Turbidity Compared to the State Standard 13.1 Wastewater Treatment FTEs per Million Gallons Treated Daily: Washington Suburban Sanitary Commission and Peer Group of Service Providers 13.2 Sewer Collection FTEs per 100 Miles of Sanitary Sewer: Washington Suburban Sanitary Commission and Peer Group of Service Providers 13.3 Analysis of the Adequacy of Police Staffing and Deployment in Kansas City’s Metro Patrol Division 15.1 Expenditures Per Household and Per Capita by the City of Golden, Colorado (Adjusted for Inflation) 15.2 Tracking Differences in CPI and IPD Inflation Rates 23.1 Number of Lane Miles of Roadway That Can Be Treated with $1 Million, Using Various Treatment Options 24.1 Pareto Chart of Public Works Complaints in Fiddler 25.1 Basic Cause-​and-​Effect or Fishbone Diagram 25.2 Cause-and-Effect Diagram for Idle Public Works Employees 26.1 Flow Chart for the Purchasing and Accounts Payable Process at the City of Springfield, Massachusetts 26.2 Process for Removing Inoperable and Abandoned Vehicles in San Jose, California 26.3 Maybree County’s Current Employee Recruitment Process Requires 215 Days to Fill Vacancies 26.4 Proposed Employee Recruitment Process Will Require 128 Days to Fill Vacancies 26.5 The Current Process for Addressing Pipe Failures and Deterioration

114 131 132 135 154 159 160 161 171 172 174 190 193 286 305 311 316 320 323 326 327 329

xv

xvi

list of figur es

26.6 Proposed Process for Addressing Pipe Failures and Deterioration 27.1 Typical Control Chart for a Process 27.2 Line Chart of Time Required to Post Payroll by Month 27.3 Control Chart for Posting of Payroll Data 27.4 Control Chart for Percentage of Invoices Paid Within 30 Days 29.1 Rate of Participation in Portland (Oregon) Parks and Recreation Programs Varies by Residents’ ZIP Codes 29.2 Perceived Housing Availability for Various Household Income Ranges in Tillery (in percentages) 29.3 Perceived Magnitude of Speeding Problem, by Neighborhood in Tillery (in percentages) 30.1 Financial Condition Dashboard for Hutchison Governmental Activities 32.1 Forecasting Revenues for the Upcoming Year by Linear Regression Trend Line 33.1 The Five Stages of Best Practice Benchmarking 33.2 Benchmarking Library Circulation per Capita across Selected Cities 33.3 Benchmarking Park Accessibility across Selected Cities: Percentage of Residents within One-​Half Mile of a Park 33.4 Residential Refuse Accounts and Expenditures per Account for 10 Cities 33.5 Relationship between Number of Residential Refuse Accounts and Expenditures per Account (Correlation: 0.39) 33.6 Positive Relationship 33.7 No Relationship 33.8 Negative Relationship 33.9 Comparing Actual to Expected Expenditures for Refuse Collection among Cities with Various Levels of Contract Services 33.10 Benchmarking Public Transit Operations across Selected Cities 33.11 Scatterplot of Violent and Property Crime Rates in Selected Cities 34.1 Choosing the Right Chart or Graph

330 338 344 346 350 372 380 381 396 428 435 437 438 441 441 442 442 443 446 448 449 461

list of figure s

34.2 What the Analysts Initially Prepared to Show Fleet Department Work 34.3 What the Analysts Prepared to Better Show the Trends in Fleet Work: Column Chart 34.4 What the Analysts Prepared to Better Show the Trends in Fleet Work: Stacked Column Chart 34.5 City of Dulles Revenue by Source, 2019 34.6 What the Analysts Prepared to Better Show the Trends in Revenue: Line Chart 34.7 What the Analysts Prepared to Show Efficiency in Spending over Time

464 464 465 466 467 468

xvii

LIST OF TABLES

1.1 2.1 3.1 3.2 3.3 3.4 4.1 4.2 5.1 5.2 6.1 6.2 6.3

Applying Analytic Techniques Incidents of Arson in the Town of Sparksville: Measures of Central Tendency Reporting Performance Measures in Fort Lauderdale, Florida Reporting Performance Measures in Coral Springs, Florida Example of Estimating Fire Inspection Effectiveness Maintenance Tasks and Work Points Fire Losses in Zornig, by Fiscal Year Depicting Zornig’s Fire Loss Statistics Prescribed Sample Sizes for an Audit of Invoices Relationship between Sample Size and Precision in a Simple Random Sample Poll: 95 Percent Confidence Level Service Calls and Officers on Duty, Satterfield Police Department, Sunday, February 2 Calculating the Correlation Coefficient for Service Calls and Officers on Duty, Satterfield Police Department, Sunday, February 2 Officer Deployment-​to-​Demand Correlations, Satterfield Police Department, February 2–​15

16 29 42 43 49 52 61 65 76 77 85 86 87

list of table s

6.4 Officer Proficiency at the Shooting Range 6.5 Calculating the Expected Frequency of Cells from Knowledge of the Marginals 6.6 Actual Shooting Proficiency (“Observed Frequencies”) by Gender of Officers 6.7 Calculations for Chi-​Square 6.8 Shooting Proficiency by Gender of Officers: Raw Numbers and Percentages 7.1 Sensitivity Analysis of Student Volunteer Yield Rate Assumption 9.1 Comparing Responsiveness to Streetlight Outages in the Westchester and Scarboro Neighborhoods 9.2 Performing a t-​Test to Compare Two Groups 11.1 Work-​Distribution Chart, Vehicle Maintenance Parts Room, City of Weber 12.1 Excerpt from City of Armadillo’s Customized Standard Time Manual 12.2 Excerpt from Performance Report, City Garage, September 13–​17: Efficiency Ratings, by Mechanic 12.3 Excerpt from Performance Report, City Garage, September 2021: Efficiency Ratings, by Repair Type 12.4 Fairfax County Fire and Rescue Department Performance Compared to NFPA Standards 15.1 Selecting an Inflation Index: CPI versus IPD, 1990–​2019 15.2 Municipal Cost Index, 1999–​2020 15.3 Police Patrol: Positions and Expenditures, City of Keystone 15.4 Converting Keystone Police Patrol Expenditures for Fiscal Year 2018 and Fiscal Year 2019 to 2017 Constant Dollars 15.5 Constant Dollar Comparison of Police Patrol Expenditures, City of Keystone, 2017–​2019 17.1 Estimated Revenues and Costs for the Swimming Pool Proposal 17.2 Present Value Analysis of the Swimming Pool Proposal 18.1 Animal Control Annual Budget Expenditures 18.2 Animal Control: Revised Annual Costs 18.3 Projected Cost Savings: Sensitivity to Expected Life of Capital Equipment 19.1 Rule-​of-​Thumb Alternatives for Annualizing Capital Costs

93 94 94 95 96 102 117 118 143 153 155 156 158 192 195 197 198 198 217 219 226 231 233 248

xix

xx

list of table s

19.2 Possible Allocation Bases for Various Indirect (Overhead) Cost Assignments 20.1 Full Cost of In-​House Operation Compared to Low Bids 20.2 Go-​Away Costs Compared to Contract Costs 21.1 Supplementing Purchase Price with Lifetime Energy Costs 21.2 Formula for Life-​Cycle Costing 21.3 Applying Life-​Cycle Costing: Comparing Water Heater Options for 20-​Year Period 22.1 Interest Factors for Compounding Uniform Series Capital Recovery Factors 22.2 Equivalent Annual Worth Analysis of the Sheriff’s Office Copier 22.3 Equivalent Annual Worth Analysis of the Inspection and Permit Office Copier 22.4 Testing the Sensitivity of Interest Rates for Equivalent Annual Worth Analysis 22.5 Testing the Sensitivity of In-​Service Periods for Equivalent Annual Worth Analysis 23.1 Years of Pavement Life Gained Through Various Preventive Treatments 23.2 Equipment and Crew Requirements of Four Refuse Collection Options 23.3 Projected Annual Direct Costs of Four Refuse Collection Options 23.4 Annualized Capital Costs for Refuse Collection Vehicles and Collection Carts 23.5 Projected Annual Direct Costs of Four Refuse Collection Options 26.1 Four Common Symbols in Process Flow Charting 27.1 Decoding Control Chart Terminology 27.2 Management Performance Report: Finance Department 27.3 Days to Post Payroll by Month 29.1 Differences by Ethnicity and Race in the Use of Community Amenities in Takoma Park, Maryland: Survey Results 29.2 Support for Proposed Rugby League among Sample of Citizens 29.3 Calculating the Expected Frequency of Cells from Knowledge of the Marginals

252 259 261 264 266 267 278 278 279 280 280 287 289 291 292 293 323 335 336 345 373 375 375

list of table s

29.4 Actual Support (“Observed Frequencies”) for Proposed Rugby League among a Sample of Citizens 29.5 Calculations for Chi-​Square 30.1 Financial Flow Indicators for General Fund 30.2 Financial Stock Indicators for General Fund 30.3 Financial Flow Indicators for Governmental Activities and Business Activities 30.4 Financial Stock Indicators for Governmental Activities and Business Activities 31.1 Gender and Racial Composition of Wattlington’s Citizen Boards 31.2 Gender and Racial Composition of Wattlington’s Full-​ Time Employees 32.1 Misty Glen Revenues: The Six Most Recent Years 34.1 General Tips for Tables and Graphs 34.2 Initial, Tightly Formatted Table Prepared by Analysts to Show Municipal Staff by Department 34.3 What the Analysts Prepared to Make the Table More Readable

376 376 390 391 392 394 407 407 427 460 466 467

xxi

LIST OF WORKSHEETS

19.1 Salaries and Wages of Personnel for a Selected Program Activity 19.2 Fringe Benefit Costs for a Selected Program/​Activity 19.3 Other Operating Expenses for a Selected Program/​Activity 19.4 Overhead Costs for Selected Program/​Activity 19.5 Cost Summary for Selected Program/​Activity

244 245 246 249 251

PREFACE

Tools for Decision Making is a guide to practical techniques of analysis for local government. It features analytic methods selected for their relevance to everyday problems encountered in cities and counties—​problems faced in different form and scale at other levels of government and in other organizations as well. Most of the techniques described in this volume can be mastered quickly and easily. Several are among the simplest that the fields of industrial engineering, policy analysis, and management science have to offer. None requires advanced mathematical skills—​at least, not in the form presented here. Most of the selected techniques could be performed by hand without too much difficulty, although access to a computer will make simple techniques even easier and will be especially important in a few cases. Spreadsheet tips and helpful websites are scattered throughout the volume. Additionally, directions are provided for online access to spreadsheet templates designed for the analysis of problems similar to those described in the chapters. Many books that address the topic of analysis for decision making miss the mark for readers searching for handy techniques that can be put to immediate use on the job. Those books focus on impressive techniques that too often are difficult to grasp without extensive formal training or a major commitment to self-​directed study. Spare time for developing new skills can

xxiv

pre fac e

be a commodity in short supply for the busy practitioner who seeks timely solutions to pressing problems. Still other books come up short for a different reason. They take a scattershot approach toward the target by providing a broad overview of the issues and the promise of analysis, but they fail to deliver practical methods of immediate value. Their authors may theorize about decision making or perhaps extol the virtues of a rational approach but then confine their description of analytic methods to only two or three rarely used techniques selected for their novelty, not their usefulness. Tools for Decision Making is different. Techniques that are abstract, impractical, or demand advanced mathematical skills are not included. Instead the reader will find easy-​to-​understand, easy-​to-​apply, practical techniques of analysis especially suited to local government. Each is offered as a simple tool for the busy public manager who favors data-​based decision making over intuition. This is the fourth generation of a book that has proven popular with students and local government practitioners alike. The original appeared under the title Administrative Analysis for Local Government. The current volume is the third edition under its present title, Tools for Decision Making—​the previous editions were published in 2002 and 2009 by CQ Press. Students have liked the book’s practical orientation and choice of useful analytic techniques that could be applied in other courses and used on the job. They particularly valued the entertaining scenarios that illuminated the techniques and demonstrated their relevance. Practitioners have praised the same things, appreciating especially the succinct and easy-​to-​grasp manner in which the techniques are introduced. Each new edition has retained these proven elements, even as material has been updated and new techniques added. The book is arranged in eight parts. Parts I and VIII offer introductory and concluding comments, respectively, on the role of analysis in local government. Parts II through VI focus on analytic techniques, clustered for similarities or common purposes. Part VII addresses the important task of presenting analytic evidence. Each chapter poses a problem in the form of a hypothetical local government scenario, describes an appropriate technique of analysis, and demonstrates the use of that technique. Like its predecessors, the third edition was written with practitioners and students of public administration and local government in mind. For the classroom market, this book will be especially useful in public administration courses

newgenprepdf

pr efac e

on research or analytic methods, statistics, policy analysis, budgeting, performance auditing, financial management, urban management, urban service delivery, and productivity improvement. The original version of this book was published by the Carl Vinson Institute of Government at the University of Georgia when one of the authors, David Ammons, was a faculty member there. Initial drafts of several eventual chapters were developed as instructional material for a series of management seminars conducted with department heads and other administrative officials of Glynn County, Georgia. Graduate students Richard Huddleston, Roderick C. Lee, and James A. Russell assisted ably in that endeavor. Our discovery of a gap between methods textbooks oriented toward academic research, on the one hand, and the analytic needs of practicing administrators, on the other—​a gap that our Glynn County instructional materials filled—​led to the book. As the book has evolved through new editions, many others have assisted in different ways, including Andy Williams, Carla Pizzarella, Stacey Teachey, Paige Ammons, Greg Mavraganis, Marsha Lobacz, Alex Hess, Frank Alford, Robby Poore, Emily Hinkle, and Macie Rouse—​ all of them gratefully acknowledged here. For this edition we have trimmed the volume in some places and expanded it in others. The result is a collection of practical analytic techniques that we hope government managers and analysts will find valuable. Finally, we wish to thank our wives, Cindy Ammons and Bing Roenigk, for their support and patience as we found ourselves devoting more and more of our time to preparing this new edition of Tools for Decision Making. David N. Ammons Dale J. Roenigk Chapel Hill, North Carolina

xxv

Part I INTRODUCTION

Too many local government issues are debated and decided without the benefit of systematic analysis. The choices made in such instances often suffer from the absence of insights that careful analysis might have provided. Sometimes analysis is overlooked because officials fail to realize that helpful analysis relevant to the issue at hand could be performed. And sometimes managers mistakenly believe that all analytic techniques are so sophisticated that they are difficult to learn and apply. Without highly trained analysts on board, they assume that the analytic task is beyond their staff’s capability. In many cases their assumption is incorrect. Most local government staffs have managers, department heads, management analysts, administrative assistants, budget analysts, and others who, with just a little instruction, can quickly become capable of performing good analysis on a wide range of important issues and management problems. This volume describes a variety of practical analytic techniques that are directly applicable to local government problems. Each is presented in the context of a fictitious scenario in a hypothetical community. The techniques are easy to learn and can be mastered quickly.

2

introduction

Learning how to perform analysis is important, but simply understanding the mechanics of a particular technique is rarely sufficient to ensure a useful result. Other factors are important, too. That is why each analytic technique is presented in a scenario, where an analyst can confront practical considerations beyond the numerical calculations. The scenarios allow the techniques to be seen in action. Furthermore, tables and figures display the products of analysis as they might be displayed to local government decision makers. For analysis to be effective in a local government setting, mastery of suitable techniques must be accompanied by: • • • • •

knowledge of when to apply a given analytic technique skill in presenting analytic information awareness of the political environment in which decisions are made sensitivity to the concerns of public employees and decision makers a realistic perception of the place of analysis in a government setting

When appropriate techniques are chosen carefully, applied properly, and presented effectively, well-​reasoned analysis can play a role—​sometimes a very important role—​in local government decision making.

1 THE ROLE OF ANALYSIS IN LOCAL GOVERNMENT

Too often, important decisions in local government are made without the benefit  of careful analysis. Conclusions are drawn and choices are made in the absence of systematic assessments of alternatives and probable consequences, and often even without an assessment of service needs. Decision making in local government sometimes involves matters as crucial to a community’s well-​being as the development of long-​term strategies for economic development or as seemingly mundane as choosing among competing bids in a purchasing decision. Whether crucial or mundane, a local government decision is much more than simply the logical product of careful analysis. Good analysis does not automatically influence decisions; in fact, the recommendations in carefully prepared analytic reports often are rejected in favor of other choices. Few, if any, local government officials—​elected or appointed—​are willing to let the numbers derived from analysts’ calculations make their decisions for them. Each brings to office a point of view, an inclination to judge problems and programs from a perspective developed over a lifetime, and perhaps a vision of where the community is or should be going. Most have advisers whose

4

introduction

judgment is particularly influential. And then there is politics—​the struggle to influence the use of resources and the distribution of public benefits, the quest for control. Personal views and politics play powerful roles in local government decision making, as they should. Democratic principles require nothing less. Too often, however, the perceived dominance of personal and political factors is used as an alibi to excuse the absence of analysis in the decision-​making process. There is room in local government decision making for carefully conceived, accurately reported analysis—​not as a pat decision rule (i.e., decision making by formula) but simply as one of several influences. Frequently, the findings of a careful study will bolster the confidence of persons already inclined to favor a position supported by that research or spur to action persons already interested in an issue but undecided on a stance. Systematic analysis occasionally will reverse a person’s prior misconceptions—​but not always. The analytically supported “right” answer is not always the one chosen, and that may discourage an analyst or a manager who has devoted time and effort to the task. A few might even be tempted never to devote so much analytic effort to a problem again. But only the most naïve of analysts and analytically oriented managers would believe the fruits of their labor will sway every decision. For that matter, the same could be said about an operative whose stock-​ in-​trade is political influence rather than systematic analysis. Few of the latter would abandon their efforts because of an occasional setback; why should the former?

Analytic heritage Many of the tools for decision making described in this book are drawn from management science, industrial engineering, and, to a lesser degree, policy analysis. The first two fields, more than the third, focus on operational details. They offer an array of techniques less tied to statistics or to models requiring advanced training. Nevertheless, all three fields contribute in some manner to a variety of analytic techniques—​even to some of the simplest and most practical. Systematic analysis of operational details is most aptly traced to industrial engineering and management science. Frederick Taylor, considered

th e role of analysis in local gov ernm ent

the father of industrial engineering, was a prominent figure not only in the history of that field but also in the development of public administration. Taylor contributed much to management thought, including advocacy of methods study, time study, standardization of tools, and the use of task systems featuring bonuses for exceeding performance standards. His work influenced public as well as private management. Taylor’s notions of “scientific management” and especially his dispassionate attitude toward workers—​though perhaps less condescending than sometimes depicted—​ remain controversial. The controversy, however, need not overshadow his contribution.1 Taylor believed that the systematic analysis of a job could lead a perceptive analyst to the development of an ideal procedure for performing that job, the design of ideal tools for carrying it out, and the identification of personal characteristics desired in persons selected to perform the work. Optimum performance could be secured by careful selection, precise training, and an incentive system that rewards overachievers. His theories were supported by abundant evidence of his own success in the application of scientific management principles in industrial settings. Techniques for work analysis, equipment and facility design, and the development of streamlined procedures drawn from Taylor’s prescriptions and those of his successors continue to provide an analytic basis for applicable types of management decisions. Additionally, techniques designed to enhance rational decision making in the use of resources can make an important contribution in local government. Analyses that more accurately diagnose problems and identify their causes, that assess the deployment of resources in response to service demands, that systematically assess staffing needs, that more accurately identify costs and more carefully weigh choices in the use of resources, and that more objectively scrutinize the flow of work in organizations—​all of these hold immense promise for improving local governments and their services.

1 See, for example, Hindy Lauer Schachter, Frederick Taylor and the Public Administration Community:A Reevaluation (Albany: State University of New York Press, 1989). A brief account of Taylor’s contributions is available from Encyclopaedia Britannica online at www.britannica.com/​ biography/​Frederick-​W-​Taylor.

5

6

introduction

Resistance to analysis Increased use of analytic techniques will not be welcomed by everyone at city hall or the county offices. Systematic analysis of problems and options will threaten the status quo. Accordingly, resistance may be anticipated from the following quarters: • • • • •

persons benefiting from current policies persons who would be inconvenienced by movement away from current operational inefficiencies persons who fear new technologies or methods persons who feel incapable of understanding the results of analysis persons who dominate the decision-​making process through charisma or brokering abilities and prefer not to have their dominance threatened by the introduction of a new element in the process

Local government decisions are sometimes made on the strength of “feel,” intuition, self-​proclaimed skill as a shrewd judge of what will work, or experience gained from “more than 20 years in this job.” Where these decisions have seemed to be satisfactory, resistance to a new frame of reference emphasizing the importance of empirical evidence and systematic analysis should not be surprising. Analysts and managers who are insensitive to the reasons for resistance to systematic analysis or who are tactless or impatient with persons who adapt slowly to the introduction of analytic processes are unlikely to experience the degree of success that might otherwise be possible. The effective analyst is usually much more than a good “numbers cruncher.” Often, effectiveness is also related to the analyst’s ability to interact with and gain the cooperation of officials and employees in the program under review and to transform the analysis into a report that conveys the analyst’s message in a manner and format both meaningful and satisfactory to the audience.

Interaction with program officials and employees Analysis can be threatening to people employed in or responsible for an operation under review. Outside analysts or even department heads, themselves, who wish to analyze some aspect of their own operation are well

th e role of analysis in local gov ernm ent

advised to keep affected employees informed about the purpose of the analysis, its scope and methods, and its anticipated benefits. Analysis conducted from a central office can be especially threatening. An analyst from the mayor’s or manager’s office who just drops in on a department and begins an analysis of operations will almost inevitably be viewed with suspicion and resentment. A far better approach might begin with a meeting involving the mayor or manager, the affected department head, and the analyst who will be performing the work.2 The purpose and general procedures of the analysis could be described at that meeting, and, if applicable, reassurances to reduce the level of anxiety could be made. The mayor or manager might explain, for example, that the department’s contributions and those of the department head are highly valued, if that statement is true, and that the analysis will simply explore the possibility of enhancing or refining those contributions. Ideally, the department head and departmental personnel can be offered a role in the analysis. At minimum, their cooperation or acquiescence should be enlisted.Where departmental resistance is evident, efforts should be made to adopt a research design that is especially sensitive to managerial concerns and has as little adverse impact on day-​to-​day operations as possible—​even if that means modest losses in detail and precision. Where the response is warmer and more cooperative, major research design concessions are less likely to be necessary; modest inconveniences and minor disruptions to normal operations that often accompany thorough analytic projects will be regarded as reasonable to achieve the accuracy that all parties, including departmental officials, desire. Subsequent meetings of a similar nature should include the manager or analyst, the department head, and affected departmental employees. Once again, the purpose and procedures of analysis should be described and reassurances, as appropriate, given. Employees should be encouraged to offer suggestions throughout the process. Employees may react positively or negatively to the prospect of analysis of their work. On the positive side, workers may be pleased that their work is regarded as important by management and flattered by the attention an analyst is giving to what the workers do and how they do 2 In some cases, the manager will perform the analysis personally, and no additional analyst will be involved.

7

8

introduction

it. In what researchers call the Hawthorne effect, workers might respond to this attention by improving performance.3 Some of these performance gains, however, may subside following the end of the review or experiment. The analyst is left to sort out the effects attributable to the experimental treatment (that is, the subject of the study) and those attributable to the Hawthorne effect. This task is made simpler when the analyst considers the possibility of the Hawthorne effect from the outset and takes appropriate steps in research design to guard against it or to quantify its impact.4 On the negative side, employees may become anxious about job security or may dig in their heels behind civil service rules or union protection. Managers should address honestly any possibility that the analysis may result in revised work procedures or reduced levels of employment. Those who are able to promise that any employment reduction will be achieved through normal attrition or reassignment rather than by layoffs can often gain important benefits in employee morale and cooperation. When restrictive civil service rules and union agreements inhibit production, the impact of these rules is likely to be exposed by the analysis. The same rules and the inefficiencies they bring can render a department vulnerable to the threat of privatization. In such cases, resistance to the analysis of operations can be especially harmful. Workers and their unions must be persuaded that finding ways to improve departmental performance and thereby reduce the threat of privatization will be in everyone’s best interest.

3 The Hawthorne effect got its name from a series of experiments at the Hawthorne Plant of Western Electric in 1924. In a controlled experiment involving plant lighting, worker productivity improved even among members of the control group who received no improved illumination at their workstations. This gain was attributed to the attention lavished on the workers by the experimenters. See F J. Roethlisberger, Management and Morale (Cambridge: Harvard University Press, 1941). For contrary views on the findings and relevance of the Hawthorne studies, see H.M. Parsons, “What Caused the Hawthorne Effect?” Administration and Society 10, no. 1 (1978): 259–​283; and Richard H. Franke and James D. Kaul, “The Hawthorne Experiments: First Statistical Interpretation,” American Sociological Review 43, no. 5 (1978): 623–​643. 4 An example of a measure taken to gauge and adjust for the Hawthorne effect in medical research is the use of placebos—​for example, sugar pills having no medicinal value that may nevertheless induce improved conditions in the control group of an experiment. To be judged effective, the actual test medication must produce gains greater than those experienced by patients taking the placebo.

th e role of analysis in local gov ernm ent

Ideally, the analysis will proceed openly and cooperatively, with program personnel reviewing and offering insights on the analyst’s preliminary findings. Difficult circumstances occasionally preclude the level of openness most conducive to good administrative analysis. When that happens, the likelihood of errors in observation or interpretation is increased, and the analytic product often suffers. Even in the most trying circumstances, the analysis should be conducted with integrity and courtesy. When the analysis has been completed and the recommendations are firm, program personnel should receive a briefing prior to the release of the formal report, if possible. Understandably, various aspects of the analytic process—​ observing, measuring, questioning, and perhaps recommending change from the status quo—​ can be threatening to employees. Program or operational criticisms will almost inevitably be taken personally, even when they are not intended to be. Courtesy, sensitivity, and common sense should govern the analyst’s actions throughout the process.

Presentation of findings The interpersonal dimension of administrative analysis is relevant to more than just relations with program personnel. Effective analysts also pay attention to their relationship with the recipients of analytically based recommendations. The intended consumers of the analytic product—​local government decision makers—​can be turned off by a technical report that describes the results of highly sophisticated analysis in terminology better suited to a gallery of statisticians than to a more general audience. Analysts who are more intent on improving local government operations and policies than on demonstrating methodological prowess should be attentive to the need for understanding and acceptance at both ends of the process—​that is, the understanding and acceptance of program personnel and decision makers. They should report their analysis in an accurate and comprehensible manner that, ideally, captures the interest of decision makers. The presentation of research findings should be as clear and concise a reflection of the analysis as possible. Important documentation, including a brief description of research methods, should be included. Less important material should be summarized or excluded. Few decision makers appreciate the tendency of some analysts to dump all the information they collected into their reports. Vital information should be included in the body of the

9

10

introduction

report, important supporting information should be placed in an appendix, and superfluous information should be banished altogether. Attaching a one-​page executive summary to the front of the report is often a good idea. The use of graphics to demonstrate major points is a helpful and practical method of presentation. The adage that a picture is worth a thousand words applies to charts and graphs as well. A clearer and more lasting impression might be produced by incorporating pie charts showing how resources are allocated, bar charts demonstrating the growth of sales tax revenue relative to property tax revenue, line graphs showing trends in intergovernmental revenues over time, or scatterplots to show relations between spending and outcomes, for example, than would a narrative or dense table of numbers alone. Graphics should be used judiciously, however, highlighting only particularly important points or findings.

The role of analysis Doing everything right—​developing a good analytic design, interacting effectively with program personnel, conducting careful analysis, and preparing a solid report—​still does not guarantee success. Would-​be analysts who hope to dominate the decision-​making process on the strength of their work will almost inevitably be disappointed. Analysis should be an important component of the decision-​making process; it may occasionally even dominate. Often, however, it will be nothing more than a counterbalance to less rational arguments that, despite evidence to the contrary, continue to be given credence. Analysts must remember that people—​not numbers—​make an organization go. A wealth of behavioral science research tells us that the human factor, the power of motivation, the importance of morale, the pride of personal responsibility, and the thrill of achievement are factors that must not be overlooked in the design of organizational change. Human response—​from the lowest level of the organization to the highest—​cannot be factored out. Compassion, compromise, tradition, and emotion all have a place in public decision making, as they should. But systematic analysis should have a place as well. Decision makers can weigh the various elements and make their choices accordingly—​perhaps sometimes ignoring, while ideally never being ignorant of, analytically based recommendations.

th e role of analysis in local governm e nt

Focus of this book Government managers whose own backgrounds and whose staff members’ backgrounds do not include extensive training in formal policy analysis or industrial engineering but who nevertheless wish to upgrade the analysis performed in their organization will find help in this book. The techniques presented are practical and easy to learn and apply. Advanced mathematical skills are not necessary. None of the techniques requires extensive study or imposes time constraints that would make it impractical for quick application. With this volume, an enterprising manager could simply point to a given chapter and tell an administrative assistant, “I want you to do this kind of analysis” to evaluate a proposal or an existing program. For analysts wanting to elevate their game in a simple and practical way, there is no more powerful and readily available tool than the spreadsheet found on most computers. While most, if not all, of the calculations in this book can be done by hand or with a calculator, spreadsheets expand what analysts can do in scope and complexity. They can simplify revisions. Spreadsheets allow the analyst to do more with the mounds of data that are generated as part of the normal operations of all governments. Throughout this book the use of spreadsheets will be guided by simple instructions found in boxes labeled “From the electronic toolkit.” In addition, templates for various techniques are available for download (https://​toolsfordecisionmaking.sog.unc.edu) to help the analyst move quickly from this book’s explanation of an analytic technique to a usable spreadsheet applying the technique. Boxes 1.1 and 1.2 offer instructions to get started on entering data, making calculations, and creating graphs with Excel.

FROM THE ELECTRONIC TOOLKIT BOX 1.1  ENTERING DATA IN A SPREADSHEET Some readers will prefer to perform the various calculations prescribed in this book by computer rather than by hand or pocket calculator. For those readers, the boxes labeled “From the electronic toolkit” provide instructions and helpful hints.

11

12

introduction

Many administrators and prospective administrators who use computers in their work or studies are acquainted with spreadsheet programs. Because Microsoft Excel is among the most popular of these programs, the instructions found in this and many subsequent boxes will help readers apply analytic techniques using that spreadsheet program. This box is an introduction. The first step is to enter some data that later can be analyzed or depicted in the form of a graph. To open Excel, either double click on the Excel icon on the Office tool bar appearing on your computer screen or select the Excel option from the Programs choice on the Start menu. A menu or “Ribbon” to use Excel language appears at the top, but most of the screen will consist of a grid of empty rows and columns. To insert data in the grid, click on one of the boxes or “cells” and simply type in the information. Most commonly information will consist of text or numbers. Organizing this information in a structured form is desirable. Consider each column to be a variable (for example, population, number of arrests, or number of business licenses issued) and each row to be an observation. Each cell may be identified by its column letter and row number. For example, the first cell is in column A and row 1; therefore, it is named cell A1. Suppose that you wish to analyze data from several cities. Your entry in cell A1 might be the label “City” with the cells A2 through A100 reserved for the names of various cities for which data are available. Column B might be population, so cell B1 would have the label “Population” and the actual populations for each respective city would be listed in cells B2 through B100. Alternatively data on arrests by month might be set up with “Month” in A1 followed by January through December in cells A2 through A13 and the corresponding arrest totals would be listed in Column B. Once data are in the spreadsheet, formulas can be set up to make calculations from the entered data. Formulas are entered in the same way that data are inserted in the grid. Type the formula directly into a cell; the computer will do the calculation and insert the answer. In order to indicate to the computer that a formula is being entered, rather than simply more data, the user must place an equal sign (=) before formulas. Users can simply type the formulas in the cell to get the desired calculation, but it is also possible to build formulas and make use of functions by going to “Formula” on the menu and using the various help tools to build and audit formulas. Functions are predefined formulas that greatly simply calculations. For example the SUM function will add up a selected range

th e role of analysis in local governm e nt

of cells rather than having to add the plus symbol between every cell used. When seeking to use functions in Excel, the user can also click on the fx in the formula toolbar above the sheet and use the help function to select and correctly build available functions. With hundreds of functions to choose from, beginning users should make use of the Formula menu to help make sure formulas and functions are used appropriately. But as users gain experience they will quickly find they can type formulas and functions into cells directly. Users should be aware that there are multiple versions of Excel and updates are added every few years. There can be small differences in features. Furthermore, the versions available for Macs are not always the same as those for PCs. Still, the basic functionality is consistent, so the instructions here should work on all versions. When applied using Excel, some of the tools and techniques described in this and subsequent chapters will call for the use of the Data Analysis Option. All versions of Excel have this add-​in available, but it is not typically installed as part of the usual Excel set-​up. Instructions for installing the Data Analysis Option, if it is not already installed on the user’s computer, are provided in Appendix F. Because it does take a small amount of computer memory, activating it is not recommended until a user knows they want to use its features. Analytic techniques using these features will be noted in subsequent chapters.

FROM THE ELECTRONIC TOOLKIT BOX 1.2  CREATING LINE CHARTS AND BAR CHARTS No doubt, many readers have already discovered that spreadsheet programs, such as Microsoft Excel, simplify the creation of line charts and bar charts and make them look professional. To make a chart using Excel, begin by entering the data in the spreadsheet as described in Box 1.1. Then, highlight the data (“select” the data, in Excel terminology) and display the Insert menu by clicking on “Insert” from the menu options at the top of the screen. Choose the type and subtype of chart that are appropriate for the data. The most useful chart types will probably be the Column, Line, Pie, and Bar options. Click to choose a type and display the chart.

13

14

introduction

After the chart is displayed, take a moment to make sure that the correct data were selected and are represented correctly in the chart. For example, if the data of interest are entered in A1 through C13 with Row 1 containing labels describing what is in each column (e.g., Month, Arrests, and Crimes), column A would have month labels (Jan, Feb, etc.) while columns B and C would have monthly data for the two different data series (Arrests and Crimes). In this example if you chose a clustered column chart, the chart would have two columns for every month. If labels for the months and labels for the two data series are part of the selected data, Excel will automatically add these to the graph. Excel’s graphing tools provide a high degree of flexibility for creating graphs with many features. But first learn how quickly basic charts can be created. If the graph type selected does not convey the message appropriately, the graph type can simply be changed rather than having to create a new graph from scratch. This can be done with a right click of the mouse or in the Chart Design menu at the top of the screen. Unless your version of Excel predates version 2013, selecting a range or block of cells will automatically display a small icon for what Excel calls the Quick Analysis tool at the bottom right corner of the range selected. This tool will help in creating a graph even more easily. It will also insert basic functions such as summing rows or columns of data. Once a graph is created nearly every element in the graph can be modified or changed, including the axes, titles, gridlines, and scales, as well as colors and fonts. Once a graph is created and then selected, Excel provides two additional submenus to the menu at the top: Chart Design and Format. These two submenus are not available until the user selects a graph. Use these menus or simply click on an item in a given graph (the horizontal axis or the legend, for example) and the user can either right click on the mouse or a separate window on the right side will pop up detailing what can be done to that particular chart element. The potential choices are quite extensive but can be ignored in the beginning, as the basic features are typically enough to create sharp charts. As a user develops skills, it becomes possible to create one’s own graph templates making it very easy to create a custom look without changing every element each time a new graph is created. Graphs are placed by default on the same spreadsheet as the data. But graphs can be placed on their own tabs, if desired. Usually, choosing to keep the graph in the spreadsheet is the better option because it can be more easily modified in the spreadsheet and it can be used to create

th e rol e of analysis in local governm e nt

reports mixing tables and graphs easily. However, to move the chart to its own sheet, right-​click in the chart and choose “Move Chart.” In the “Move Chart” textbox, choose whether it should be a “New sheet” or remain as an “object in” the current sheet.

This book is not a compendium of all the best techniques of policy analysis, management science, and industrial engineering. Although the methods described are among the most useful for the analysis of local government problems, many other excellent techniques have been excluded here simply because they require more specialized skills or more extensive study time than most busy managers or administrative assistants have or are willing to invest.5 Local governments wishing to apply these more sophisticated techniques to the analysis of their problems may find that the best way to do so is by hiring analysts with advanced skills or by using specialized consultants for such purposes. In contrast, the analytic techniques featured in this book are potent, practical, and can be mastered by their current staff. These techniques are presented in a manner that is simple, direct, and designed to minimize the dread many readers feel for methods textbooks. Table 1.1 lists the analytic techniques included in this book and briefly describes how each is used. The table also presents a sample of the wide variety of practical local government questions that these techniques can help answer. Analytic techniques sharing common characteristics are clustered in Parts II through VI of the book. Part II covers many of the basics of statistical analysis but includes much more than simple descriptive statistics. Part III includes techniques useful for analyzing service demands and an organization’s capacity for meeting those demands, while Part IV addresses analytic techniques pertaining to costs. Part V focuses on techniques associated with process improvement and Part VI includes a collection of techniques ranging from the analysis of citizen survey data to the analysis 5 Analytic techniques more advanced than those featured in this book include cost-​benefit analysis, data envelopment analysis, techniques for identifying optimum siting of facilities and optimum routing, predictive analytics using advanced algorithms, queuing models, simulation techniques, and decision analysis under conditions of uncertainty.

15

newgenrtpdf

16

Analytic technique

Chapter

Use

Responding to questions like these…

Benchmarking

33

Detect performance gaps and identify, through comparison with top performers, ways to close the gaps

Binomial statistics

31

Determine the likelihood of an occurrence by chance or as the result of an underlying influence—​perhaps bias

Break-​even analysis

8

Determine the volume of activity required to recoup the organization’s investment in a revenue-​producing or cost-​saving initiative

Cause-​and-​effect diagram Central tendency statistics Chi-​square statistic

25

Identify factors that contribute to the success or failure of a given process Identify the norm, typical, or average case

How does the pet adoption rate at the animal shelter compare with applicable standards or with the rates of top shelters? What operating practices account for the superior results of top animal service operations and which of these are adaptable for local use? Is the composition of citizen boards and commissions representative of the community’s racial and gender distributions? What about the local government workforce? Would a water slide attract enough additional swimmers and admission revenues to the pool to justify the expenditure? If the city print shop’s capability is enhanced by a major upgrade of equipment, some of the currently contracted jobs could be handled in-​house. How much work would have to be shifted in-​house to break even on the upgrade? We keep getting complaints about street crews standing around. Why are the workers idle so much of the time? What’s the typical police salary in this city? About how many refuse complaints are received daily? Do the residents of different neighborhoods feel about the same regarding the quality of police services? Are employees who participate in the tuition reimbursement program more likely than non-​participants to remain with the county government at least five years?

2 6 & 29

Detect statistically significant relationships between nominal variables

introduction

Table 1.1  Applying Analytic Techniques

16 & 17

Calculate the cumulative effect of interest on an investment

“Constant dollar” calculations

15

Control chart

27

Factor-​out the effects of inflation when examining changes in revenues or expenditures Separates signals from noise in the typical variation of performance numbers

Correlation

6

See the extent to which increases in one variable tend to be accompanied by increases (or decreases) in another variable

Cost-​effectiveness analysis

23

Analyze the cost differences among a set of options that will provide equivalent benefits or analyze the different benefits among options having equivalent costs

Demand analysis

10

Identify patterns of demand for a given service

If the city invests $50,000 annually for the next five years at 4 percent interest as a strategy for funding a special project, how much money will be available? Controlling for inflation, have expenditures increased or decreased for various services? By how much? Performance measures bounce around from month to month for a variety of reasons. A lot of the variation is beyond the control of the workers and supervisors. How can I tell the difference between a month of commendable performance and a poor-​performance month? How closely does the pattern of lifeguards on duty coincide with the number of swimmers at the pool? How closely does the demand pattern at the customer service desk correlate with the number of clerks on duty? Considering several alternatives for maintaining landscaped medians at a specified level of quality, which option offers the best value? Given a specified level of resources for combating the loss of books from the library’s inventory, which of three options is most likely to solve the problem? What are the hourly consumption patterns for water or electricity? What strategies might help shave the demand peaks? What are the demand patterns for other local government services? How well does the pattern of resource deployment match the demand pattern? (continued)

th e role of analysis in local gov ernm ent

Compounding

17

newgenrtpdf

18

Analytic technique

Chapter

Use

Responding to questions like these…

Depreciation

18

Calculate annualized costs for equipment and other capital items

Discounting

17

Calculate the present value of resources that will be received in the future

Dispersion, measures of

2

Identify how tightly clustered a set of cases is

Exponential smoothing

32

Financial condition analysis Forecasting

30

Forecast future values using a formula that relies on previous values and adjusts for errors in projections of prior periods Identify financial strengths, weaknesses, and emerging challenges

How much of the unit cost of the service we provide is attributable to the cost of the equipment we use? How do our capital costs compare to those of other organizations doing similar work? If the city offers an industrial recruitment incentive with a projected long-​term payout, is it a good deal for general taxpayers? How widely spread are property values in this community? Is there much variation in the number of calls for police service from one night to another? Based on current trends, what population total should we anticipate for the near future? What about revenues? Program attendance? Tourism? Calls for service? Is the local government in relatively good shape with regard to its debt burden? Solvency? Liquidity?

Full costs of a program, identifying

19

32

Predict future outcomes (e.g., revenues and other outcomes) by some combination of expert judgment, trend analysis, and deterministic and econometric techniques Incorporate into total costs not only direct costs but also indirect and other costs that often fail to be included in the reporting of program or activity expenditures

What amount of revenue should we expect for next year and the year after that? What should we anticipate as our population in the near future? What can we forecast as program attendance? Tourism? Calls for service? What does it cost to repair a pothole? To respond to a fire alarm? To what extent are the full costs of the building inspection program recovered by fees?

introduction

Table 1.1 Cont.

16 & 17

See what today’s resources, if invested, will be worth in tomorrow’s dollars

“Go-​Away Costs” (privatization decisions)

20

Investment calculations

16

Lease-​buy analysis

22

Compare the cost of contract services to a corresponding reduction of in-​house expenditures if a local government chooses to no longer produce the service itself (this technique excludes from consideration any overhead and other costs that will continue despite a contracting decision) Calculate yields from simple or compound interest; calculate interest rates; calculate yields from a series of even, annual investments (annuity) Calculate the financial advantage or disadvantage of purchasing an item rather than leasing it

Life-​cycle costing

21

Consider all costs associated with an item over its useful life when deciding whether to acquire it, rather than purchase price alone

Moving average

4

Smooth short-​term fluctuations in performance statistics by reporting averages over longer periods of time

If the city invests $70,000 annually for the next four years at 4 percent interest as a strategy for funding a special project, how much money will be available? [see “compounding” above] How much will the local government save, if anything, by contracting refuse collection services? Custodial services? Street maintenance? Park or right-​of-​way mowing? Other services?

How well are the county’s investment strategies working? Can sufficient resources be accumulated from systematic set-​asides to meet our capital program needs? Would the county be ahead financially to buy its sedans and pickups or to lease them? What about leasing other equipment? Should the county convert its gasoline-​powered fleet to an alternate fuel? Which pumper should the fire department and fleet maintenance recommend? What are the life-​ cycle ramifications of purchasing smaller-​capacity, gasoline-​powered refuse collection trucks rather than larger, diesel trucks? The city could report monthly fire incident and fire loss statistics or it could report as a single figure the monthly average for the last 12, 24, or 36 months. Which option would be better? (continued)

th e role of analysis in local gov ernm ent

Future value

19

newgenrtpdf

20

Analytic technique

Chapter

Use

Responding to questions like these…

Opportunity costs

17

Pareto chart

24

Would a multimillion-​dollar expenditure for a general aviation airport be a wise investment for the community? Would the development of an industrial park be a better choice? Where should the city concentrate its efforts in reducing the volume or significance of citizen complaints?

Performance measurement

3

Help sort out choices among possible resource uses by comparing the benefits of one choice to the opportunities forgone by failing to select another option Identify instances in which a small proportion of cases are causing disproportionate problems or expenses Gauge the quantity, quality, efficiency, effectiveness, and equity of performance by a work unit, department, or program

Performance standards

12

Judge the adequacy of current services or the proficiency of current performance

Present value

17

See what a future accumulation of resources would be worth in today’s dollars

Process flow charts

26

Identify and analyze each step of a regularly performed process in an effort to eliminate unnecessary steps, reduce delay, and streamline procedures

How well is the parks maintenance function being performed? Is the quality satisfactory? What about the level of efficiency? Does performance compare favorably with relevant benchmarks? Are services equitable across neighborhoods? Has the department reported its performance to the city council and to the citizenry? How does local performance compare with published standards for grounds maintenance, custodial service, or vehicle repair? If the city offers an industrial recruitment incentive with a projected long-​term payout, is it a good deal for general taxpayers? Why can’t we recruit and hire new employees more quickly? Why does approval of permit applications take so long? Why can’t account clerks process more transactions per day? Why does it take so long to dispatch emergency calls?

introduction

Table 1.1 Cont.

32 & 33

Describe the relationship between two or more variablesa or perhaps predict changes in one variable based on the actual or projected behavior of other variables

Sampling, randomization

5

Learn about a larger population by studying a carefully selected subset

Sensitivity analysis

7

Staffing analysis

13

Test a given analytic result’s sensitivity to various assumptions embedded in the analysis by repeating the analysis using different assumptions Examine the adequacy of current or proposed staffing of a given function

Staffing factor calculation

14

Calculate the staffing required to provide uninterrupted coverage of essential positions

How much is the level of participation in the city’s recreation programs influenced by demographic factors as opposed to things like scheduling and participation fees? Given current economic factors and the community’s demographics, what rate of tax delinquencies may we forecast? What level of general fund revenue should we project? Are citizens generally satisfied with recreation services? What do they think about the city’s traffic engineering efforts? Do the citizens support local efforts to find a site for a local airport? How much of an account clerk’s workday is devoted to handling calls (work sampling)? What is the transaction error rate (audit sampling)? How much would the analytic result differ if the analyst had assumed a higher or lower rate of interest? A longer or shorter useful life for the equipment? A larger or smaller cost-​of-​living increase for employee wages? Is the recreation department understaffed, as the recreation director claims? Shouldn’t we have as many police officers per 1,000 population as other cities? The swimming pool will be open 72 hours per week this summer. How many lifeguards should the recreation department hire to be certain that it has a qualified person on each of the three lifeguard towers whenever the pool is open? (continued)

th e role of analysis in local gov ernm ent

Regression

21

newgenrtpdf

22

Analytic technique

Chapter

Use

Responding to questions like these…

Surveys of citizens or customers

28 & 29

Learn the views of citizens or customers directly, practically, and reliably

t-​test

9

Determine whether the differences in average performance across work units or neighborhoods are meaningful

Time & motion and time allocation studies Work distribution analysis

13

Examine systematically the actions of workers as a means of analyzing staffing and identifying improvements in work processes

11

Identify the time devoted by each member of a work unit to various tasks as a means of checking the alignment of priorities and resource allocations, qualifications and tasks, degree of specialization, and backup capabilities

Are citizens satisfied with the quality of city services? Should we survey by mail or by another method? How should the questions be posed? How should the responses be analyzed? Would using a focus group be a more suitable method of securing the feedback we need? Detailed work reports show variation in the patterns of service provided to all neighborhoods, but the performance average reported for a neighborhood with affluent residents is slightly more favorable than for the neighborhood with the city’s least affluent neighborhood. Is this small difference insignificant or is it meaningful? Are our local government functions properly staffed? Are current procedures efficient or are they wasteful? Do current procedures produce unacceptable amounts of idle time? Is this work unit directing sufficient time of skilled employees to top-​priority tasks? Is the degree of specialization appropriate? When absences occur, can we still cover all tasks? Are employee skills and tasks suitably matched?

a  Classically, all variables in a regression analysis would be continuous, but, practically, the independent variables can be of other types as well.

introduction

Table 1.1 Cont.

th e role of analysis in local gov ernm ent

of benchmarking data. Part VII offers recommendations for presenting analytic evidence. Beginning with Chapter 3, each analytic technique is presented and applied in the context of a brief, hypothetical case describing a common local government problem in a fictitious community. As in real life, analysis of these hypothetical problems does not always dictate a solution, but often the analytic process illuminates the strengths and weaknesses of various options.

References Franke, Richard H., and James D. Kaul. “The Hawthorne Experiments: First Statistical Interpretation,” American Sociological Review 43, no. 5 (1978): 623–​643. Mee, John F. “Frederick W. Taylor: American Inventor and Engineer,” Encyclopaedia Britannica online at www.britannica.com/​biography/​ Frederick-​W-​Taylor Parsons, H. M. “What Caused the Hawthorne Effect?” Administration and Society 10, no. 1 (1978): 259–​283. Roethlisberger, F. J. Management and Morale. Cambridge: Harvard University Press, 1941. Schachter, Hindy Lauer. Frederick Taylor and the Public Administration Community: A Reevaluation. Albany: State University of New York Press, 1989.

Suggested for further information Aft, Lawrence S. Work Measurement and Methods Improvement. New York: John Wiley & Sons, 2000. Berman, Evan. Performance and Productivity in Public and Nonprofit Organizations, 2nd ed. Armonk, NY: M. E. Sharpe, Inc., 2006. Freivalds, Andris, and Benjamin W. Niebel. Niebel’s Methods, Standards, and Work Design, 13th ed. New York: McGraw Hill, 2014. Koehler, Jerry W., and Joseph M. Pankowski. Continual Improvement in Government: Tools and Methods. Delray Beach, FL: St. Lucie Press, 1996. Meier, Kenneth J., Jeffrey L. Brudney, and John Bohte. Applied Statistics for Public and Nonprofit Administration, 9th ed. Stamford, CT: Cengage Learning, 2015.

23

24

introduction

Miller, Larry, and Robert Dolan, Jr. Government Analytics for Dummies. Hoboken, NJ: John Wiley & Sons, 2012. Newcomer, Kathryn E., Harry P. Hatry, and Joseph S. Wholey, eds. Handbook of Practical Program Evaluation, 4th ed. Hoboken, NJ: Jossey-​Bass, 2015. Summers, Michael R. Analyzing Operations in Business: Issues, Tools, and Techniques. Westport, CT: Quorum Books, 1998. Weimer, David L., and Aidan R. Vining. Policy Analysis: Concepts and Practice, 6th ed. New York: Routledge, 2017. Welch, Susan, and John Comer. Quantitative Methods for Public Administration: Techniques and Applications, 3rd ed. Long Grove, IL: Waveland Press, 2006.

Part II THE BASICS

The fundamentals of practical analysis are … well, pretty basic. To do practical analysis you will need to: •

• • •

• • •

understand basic descriptive statistics—​things like the central tendency statistics of mean and median that show where the data are centered, and measures of dispersion that show how much the data points are scattered (Chapter 2) know how to compile and use information about the program or organization’s performance (Chapter 3) know how and when to smooth otherwise erratic data movements by using moving averages (Chapter 4) recognize the value of a complete set of observations, but also appreciate the need for sampling (Chapter 5), when collecting a complete set is impractical master a few widely applied statistical techniques for data analysis—​ techniques like correlation, chi-​square, and t-​tests (Chapters 6 and 9) appreciate the importance of testing the assumptions being made in data analysis (Chapter 7 on sensitivity analysis) know how to predict the point at which a good idea will move from red-​ink status into the black (Chapter 8 on break-​even analysis)

Learning these things at the outset provides a solid foundation for a wide variety of practical analytic techniques still to come.

2 CENTRAL TENDENCY AND DISPERSION

Perhaps few local government managers and administrative assistants would call themselves expert statisticians or world-​class analysts, but through daily exposure most are familiar with many of the basics of analysis. They could easily draw on that familiarity to examine systematically the problems and opportunities they confront. Having the ability to apply even the simplest methods has become increasingly useful as the public becomes more and more accustomed to receiving information that is the product of analysis. No one can escape the daily barrage of facts and figures that represent one form of analysis or another. Unemployment and inflation rates, school dropout rates, Dow-​Jones averages, Nielsen ratings, Gallup polls, election night projections, even the label on the side of the cereal box stating how many calories and how much of the recommended daily allowance of various nutrients is contained in a single serving—​all of these expose us to the analyst’s stock-​in-​trade and serve as a subtle primer for understanding and applying basic analytic techniques. Local government analysts are not likely to be asked to track the stock market, to gauge the popularity of television programs, or to analyze the ingredients of

28

the basics

food items. But the regular reporting of these and other products of analysis warms all of us—​including the potential analyst—​to the use of numbers and charts to make important points. Perhaps without even realizing it, we absorb useful lessons. Just seeing how the media and advertisers use and display the results of various techniques makes people more analytically creative, more willing to try simple analysis, and more receptive to the analysis of others. Simple exposure to the presentation of facts and figures in the popular media, sometimes accompanied by a sentence or two about the methods used to arrive at them, cannot replace advanced training as a means of preparing local government analysts, but it does provide a starting point. From that common point, every manager and administrative assistant is at least capable of improving the quality of written and oral reports by including the simplest of descriptive statistics—​for example, relevant percentages, trend lines, averages, and perhaps more detailed information from a sample of the whole. Incorporating information of this type will not be difficult; after all, administrators have seen similar uses of data over and over again. Nevertheless, a word of caution is in order. While it is true that many administrators fail to recognize the analytic capability they already possess, it is also true that others make unfortunate mistakes because they assume they know more than they actually do about even the simplest aspects of analysis. For example, a manager who uses the arithmetic mean to describe the “typical case” in a group has selected the most popular statistic for that purpose but not necessarily the best. Similarly, the manager who proudly reports results from what he calls a “random sample” may be embarrassed when a more methodologically sophisticated critic informs him that “random” does not mean “haphazard” (see Chapter 5). Devoting a little time to the basics to supplement the foundation already acquired through exposure to the popular media is a wise investment. In this chapter, two basic topics important to practical analysis are examined. These are the most elementary of descriptive statistics—​measures of central tendency and measures of dispersion.

Central tendency “How many cases of arson do you have in this town in an average year?” That question seems simple enough. Just add all the cases over a period of years, divide by the number of years, and the result is the yearly average—​ technically referred to as the arithmetic mean. Although the mean is the most

c e ntral t e nde ncy and disp ersion

commonly used measure of central tendency, two other measures—​the median and mode—​may be as informative or even preferable in some circumstances. A questioner asking “what is average” may actually be seeking to know what is typical. In such cases the median or mode might be a better representation of what is typical. Suppose, for example, that the town of Sparksville is a quiet community that has rarely experienced more than two cases of arson in a year. In 2018, however, an uncharacteristic rash of arsons occurred, leading to the arrests of three youths. Calculating the yearly mean for the five-​year period from 2017 through 2021 yields an average of 2.6 arsons per year (Table 2.1). While mathematically accurate, that figure surpasses the actual number of arsons in every year except 2018, when 10 of the period’s 13 cases occurred. If the questioner was really interested in knowing the number of arsons in a typical year in Sparksville, the median or mode would probably be a measure preferable to the mean. The mode is the number appearing most frequently in an array of numbers. In this case, zero appears twice, so the mode is 0. It could be argued that in a typical year in Sparksville there are no incidents of arson. The median value for an ordered array of numbers is the midscore of the distribution when there is an odd number of observations—​that is, there are as many scores in the array with values higher than the median as there are scores with lower values. In the case of Sparksville, the median is 1—​for there exist two years with higher numbers of arson and two years with a lower number. It could be argued that one arson per year is most representative of reality in Sparksville. For situations where the number of

Table 2.1  Incidents of Arson in the Town of Sparksville: Measures of Central Tendency Yeara 2018 2019 2021 2017 2020 Five-​year total

Incidents of arson 10 2 1 0 0 13 13/​5 = 2.6

}

Median Mode Mean

a The years have been arranged in descending order of arsons rather than chronologically in order to demonstrate how the median value is found.

29

30

the basics

observations is an even number—​thereby putting two observations at the centermost point of the array—​the median is determined by calculating the mean of these two centermost values.

FROM THE ELECTRONIC TOOLKIT BOX 2.1  FINDING THE MEAN, MEDIAN, AND MODE Using Microsoft Excel, the mean, median, and mode of a set of numbers can be found without manual calculations or sorting of the data. First, enter the data in column A or another column of your choice. (For directions on opening Excel and entering data refer to Box 1.1.) To find the mean of the data, use the AVERAGE function. Suppose the data were entered beginning in cell A1 and down to cell A8. To find the mean, enter “=AVERAGE(A1:A8)” in the next cell or any cell of your choice. (An alternative to typing this instruction would be to choose “Formulas” from the menu options at the top of the screen, and select the “Average” function from the statistical function option under “More Functions.” This method includes prompts for the data needed in the formula. The (fx) symbol also appears under the menu in what is known as the formula bar. This feature will list the available functions, including “average.”) After pressing enter, the mean will appear. To find the median of the data, use the MEDIAN function. Enter “=MEDIAN(A1:A8)” in any cell and the computer will determine the median of the data entered from cell A1 to cell A8. To find the mode of the data, use the MODE function. Enter “=MODE(A1:A8)” into any cell and the computer will report the mode of the data. It is possible to have more than one mode in some sets of data. While the old MODE function still works, two new versions of the mode function are available: MODE.SNGL and MODE.MULT. These can be used to find the single most common mode and multiple modes, respectively, where they exist. When MODE or MODE.SNGL are used, if there are multiple modes the spreadsheet will report the first occurring mode but not others. Suppose the data were entered somewhere else on the spreadsheet. Simply change the input to fit the addresses of the data. For example, if the data of interest were entered in cells C14 to C23, simply change the formulas to “=AVERAGE(C14:C23)” or “=MEDIAN(C14:C23)” or “=MODE(C14:C23).”

c entral t end ency and dispe rsion

To find mean, median, and mode simultaneously in Excel, use the Data Analysis option from the Data menu. (For instructions on installing the Data Analysis Option, if not already available, see Appendix F.) Select Descriptive Statistics from the choices provided by the Data Analysis pop-​up box. In the Input box enter the range of the data; for example, “$C$14:$C$23” designates cells C14 through C23. (These cells may also be designated by highlighting them using the mouse.) In the Output options box, check the box beside Summary Statistics. Click “OK” and a new sheet will appear with a chart containing the mean, median, and mode, as well as other statistics, including the standard deviation.

The choice of a particular measure of central tendency should be made carefully, for each has strengths and weaknesses as a representation of the central value of a set of numbers. The most popular measure of central tendency, the mean, is perhaps the best known by most audiences and is also the measure called for most often in statistical formulas. Nevertheless, it is more vulnerable than the median or the mode to the distorting effect of extreme outliers—​values far greater or far smaller than most of the others in the group.1 For example, including the salary of a highly paid supervisor could substantially inflate the mean salary of an otherwise low-​paid unit and leave the impression that the workers are relatively well paid. The median salary of the work unit is less influenced by outliers and would probably be a more appropriate choice as the measure of central tendency in that case.

AVOID THIS COMMON ERROR BOX 2.2  BEWARE OF AVERAGING THE AVERAGES When an analyst has subgroup averages and needs the average for the combined group, it is tempting to simply calculate the mean of the subgroups’ averages. The resulting number, however, probably would miss the mark.

1 In statistics terminology, a distribution of observations is said to be “skewed” when there are more extreme cases in one direction than the other. These outliers have a greater influence on the mean of the distribution than on the median.

31

32

the basics

Consider this example. Suppose you are asked to report the average daily workload of the city’s inspectors. You know the average for the three inspector subgroups in the department and might be tempted to simply add those three numbers and divide by three. Doing so would be simple, but it would fail to account for the unequal numbers of inspectors in the three subgroups. The city’s eight building inspectors perform an average of 16 inspections apiece each day; its two plumbing inspectors, 14 inspections apiece; and its three electrical inspectors, 12 inspections apiece. The breakdown is shown below:

Building inspectors Plumbing inspectors Electrical inspectors

Subgroup average

Individual daily averages

16 14 12

17, 16, 16, 18, 15, 16, 13, 17 15, 13 14, 11, 11

What is the departmental average? It is not 14, the average of the means (16, 14, 12). The mean of the three sets combined (17, 16, 16, 18, 15, 16, 13, 17, 15, 13, 14, 11, 11) is 14.77—​closer to 15 inspections per day apiece.

Dispersion Unlike measures of central tendency, measures of dispersion indicate how tightly clustered or how spread out a set of numbers is. A statistic known as the standard deviation performs this function by indicating with a single measure how much the individual numbers in a set differ from the mean value of that set. A standard deviation of zero would indicate the ultimate in tight clusters: all values in such a set would be exactly the same, so none would deviate at all from the group mean; larger standard deviations would indicate greater spread. Let us imagine that we have a year’s worth of statistics on police response times to emergencies. We are told that the average (that is, the mean) emergency response time was 5 minutes and that the standard deviation for the set of observations was 2.0 minutes. We understand the mean, but how may we interpret this standard deviation? What, specifically, does it tell us? If we assume a normal distribution, approximately two-​thirds of all observations will fall within one standard deviation of the mean and about

c e ntral t e nde ncy and disp ersion

95 percent within two standard deviations. If a police department’s emergency responses average 5 minutes and have a standard deviation of 2.0, we could expect two-​thirds of all response times for this department to lie between 3 minutes (one standard deviation quicker than the mean) and 7 minutes (one standard deviation slower). We could expect 95 percent of all response times to fall between 1 minute and 9 minutes (or within 2 standard deviations of the mean of 5 minutes). Although the formula for calculating the standard deviation may look rather complicated to persons unaccustomed to mathematical symbols, the computation is actually quite simple.2 Much to the dismay of statisticians, who correctly point out the necessity of the standard deviation calculation for many useful analytic techniques, few audiences for the practical analyses performed today in local governments insist on a measure of dispersion as precise as the standard deviation or the application of the more sophisticated techniques that call for it. For most purposes, simply reporting the range from the lowest to the highest number in the array is deemed sufficient. A slightly more sophisticated approach omits the outliers at the extreme ends of the distribution by dropping the upper and lower quartiles (that is, the top 25 percent and the bottom 25 percent of the distribution) and reporting the range of the second and third quartiles only. This interquartile range extends from the 25th percentile to the 75th percentile. Reporting the interquartile range tells the audience where the heart of the distribution lies. Both the full range and the interquartile range of a distribution may be depicted simply in a graph called a box-​and-​whiskers plot (Figure 2.1). This graph shows the highest, median, and lowest values in the distribution and highlights the interquartile range. Yet another practical way to check on and report dispersion in a set of observations is by considering the percentage of observations within a 2 The standard deviation for a sample of data (rather than the entire population) may be calculated by first finding the mean of the numbers involved, squaring the difference between each number and the mean, adding those squared values together, dividing by the number of values in the set minus one, and finding the square root of that quotient. For readers familiar with mathematical symbols and formulas, the standard deviation is as follows:

∑ (x − x )

2

s=

n −1

33

34

the basics

Figure 2.1  Range and Interquartile Range: Box-​and-​Whiskers Plot

specified range around the mean or median. The city of Coral Springs, Florida, for example, tracked both its average response time to emergency medical service (EMS) priority calls and the percentage of responses within 6 minutes. Unlike some cases where the specified range extends symmetrically from the mean (for instance, from 2 minutes quicker to 2 minutes slower than the average response time), the specified range in Coral Springs ran from 0 to 6 minutes and was not symmetrically arrayed around the mean response time. Nevertheless, even this asymmetrical range provided EMS managers and other city officials information about how tightly EMS responses were clustered within the desired range of response times. Having a high percentage within the specified range would signal a tight cluster; having a low percentage would indicate greater dispersion.

FROM THE ELECTRONIC TOOLKIT BOX 2.3  FINDING THE STANDARD DEVIATION AND THE INTERQUARTILE RANGE Once the data from a sample are entered, software like Microsoft Excel can easily compute the standard deviation and interquartile range of the sample. (For directions on opening Excel and entering data refer

c e ntral t endency and disp ersion

to Box 1.1.) To find the standard deviation using Excel, use the STDEV function. If the data of interest appear in cell A1 through cell A8, enter “=STDEV(A1:A8)” in any cell and the output that appears in that cell will indicate the standard deviation. (An alternate method for finding standard deviation is described in Box 2.1.) Finding the interquartile range is simple, too. Just use the QUARTILE function to find the first quartile and third quartile. These are the boundaries of the interquartile range. To find the first quartile for a set of data displayed in cells A1 through A8, enter “=QUARTILE (A1:A8,1)” in a cell of your choice and for the third quartile enter “=QUARTILE(A1:A8,3)” in another cell. (An alternative to typing this instruction would be to use Function choice (fx) from the toolbar.) To find the width of the interquartile range, subtract the smaller quartile number from the larger. (If you displayed the first quartile result in cell A9 and the third quartile result in cell A10, then “=A10-​A9” can be entered in a different cell to display the width of the interquartile range.) Excel has refined the quartile function, retaining the old version but adding two new options. QUARTILE.EXC (“greater than”) is the quartile function exclusive, which means that Excel will tell you the value that a given quartile is greater than. QUARTILE.INC (“greater than or equal”) is the quartile function inclusive, which means that Excel will tell you the value that a given quartile is greater than or equal to. For additional information: https://​exceljet.net/​excel-​functions/​excel-​quartile.exc-​function

Many local governments pay attention to central tendency, but relatively few give much thought to dispersion. Paying attention to both can lead to beneficial results. For example, focusing on dispersion as well as central tendency paid dividends for Coral Springs citizens after an operating adjustment designed to improve average EMS response time temporarily led to a declining percentage of responses within 6 minutes. EMS officials had observed that an assisted living facility in the community was their largest source of calls, and they reasoned correctly that moving the centrally located rescue unit to a fire station nearer this facility would improve average response time. What they had not anticipated, however, was that the improved average response time following repositioning would be accompanied by an increase in the percentage of emergency responses taking

35

36

the basics

longer than 6 minutes. Moving away from a more central location created lengthy runs to some locations. To correct this problem they decided to begin dispatching a fire pumper stationed at the centrally located fire station and staffed with firefighter-​paramedics to the relatively few life-​threatening EMS calls coming in from locations now remote to the primary rescue unit. With this adjustment—​and thanks to tracking dispersion as well as central tendency—​improvements were recorded in both measures of EMS responsiveness: “average response time” and “percentage of responses within 6 minutes.”3

BOX 2.4  OTHER APPLICATIONS OF MEASURES OF CENTRAL TENDENCY AND MEASURES OF DISPERSION Practically every analysis, it seems, calls for a measure of central tendency. How many inspections does an electrical inspector complete in a week? What is the fire department’s “control time” for working fires? How quickly does the public works department respond to pothole complaints? Each of these calls for a measure of central tendency. But which one—​mean, median, or mode? The most common choice is the mean, but that is not always the best choice.

Measures of dispersion offer important information regarding many aspects of community condition as well as local government performance. The range, interquartile range, or standard deviation of household incomes or housing values is much more revealing information than the mean family income or median house price alone. Similarly, measures of dispersion can provide reassurances of consistent quality and timeliness of local government services. If the county reports that the average property appraisal is within 5 percent of market value, can we be confident that most appraisals are within 15 percent? If the police department reports an average emergency response time of 5 minutes, should we assume that most responses are between 3 and 7 minutes? If the average sick leave usage rate among municipal employees is five days per employee per year, should we assume that the city has few, if any, problems with sick leave abuse? Measures of dispersion will tell us. 3 Ellen Liston, deputy city manager, City of Coral Springs, Florida (email communication with author, January 18, 2008).

c e ntral t e nde ncy and disp ersion

Building on the base Most managers and administrative assistants have been exposed more often than they probably realize to the products of analysis. By building on that base through the development of practical analytic skills, they can improve the quality and persuasiveness of their recommendations. The chapters that follow describe a variety of analytic techniques that will help them in that endeavor.

Suggested for further information Aron, Arthur, Elliot J. Coups, and Elaine N. Aron. Statistics for the Behavioral and Social Sciences, 6th ed. Upper Saddle River, NJ: Pearson, 2019. Eller, Warren S., Brian J. Gerber, and Scott E. Robinson. Public Administration Research Methods: Tools for Evaluation and Evidence-​Based Practice, 2nd ed. New York: Routledge, 2018. “Descriptive Statistics,” Chapter 13, 270–​290. Meier, Kenneth J., Jeffrey L. Brudney, and John Bohte. Applied Statistics for Public and Nonprofit Administration, 9th ed. Stamford, CT: Cengage Learning, 2015. O’Sullivan, Elizabethann, Gary Rassel, Maureen Berner, and Jocelyn DeVance Taliaferro. Research Methods for Public Administrators, 6th ed. New York: Routledge, 2017. The Economist Numbers Guide: The Essentials of Business Numeracy, 6th ed. New York: Public Affairs, 2014. Welch, Susan, and John Comer. Quantitative Methods for Public Administration: Techniques and Applications, 3rd ed. Long Grove, IL: Waveland Press, 2006.

Web resource For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

37

3 PERFORMANCE MEASUREMENT AND MONITORING

Performance measurement is a fundamental building block for performance management and evaluation. A set of performance measures—​also called metrics—​typically provides information about how much service was provided by a given program or department. If it is a good set of measures, it also provides vital information about the efficiency with which these services were provided, the quality and equity of services, and the effect that the services had on their recipients or on the community in general. Many cities and counties cite accountability as their chief objective in performance measurement. First and foremost they want to document their level of service in reports to the governing body and the citizenry. As important as accountability is, local governments that are determined to get the greatest value from their performance measurement system want even more. These governments use performance measures not only for accountability but also to monitor and improve services. They measure more than just workload or outputs (for example, applications processed, cases handled, or tons of asphalt laid). They also measure efficiency (for example, cost per application processed, tons of refuse collected per refuse collection employee, or lane-​miles swept

p erformanc e measur em e nt and monitoring

per operator hour). They gauge service quality and effectiveness (for example, percentage of refuse collection routes completed on schedule or percentage of fires confined to the room of origin). They check the equity of services to confirm that different neighborhoods and different service recipient groups are treated equally. They set performance targets and establish systems of regular performance feedback to department heads, supervisors, and employees.

Examples of performance measurement Most cities and counties that collect performance measures report many of these measures in their annual budget documents. Some also produce quarterly or annual performance reports that focus primarily on their measures. The budget document of the city of Fort Lauderdale, Florida, lists performance measures for each department. For instance, the budget pages for Fort Lauderdale’s water treatment and wastewater systems report these systems’ performance measures and the performance targets they are attempting to reach (Table 3.1). Many of the targets are tied to American Water Works Association (AWWA) benchmarks. The city of Coral Springs attaches symbols to its performance measures to indicate whether performance is meeting targets or falling short (Table 3.2). An upward pointing arrow indicates that performance is meeting the target. Performance measures for the development services department are shown below.

BOX 3.1  PERFORMANCE MEASURES FEATURED IN CHARTS OR DASHBOARDS Most performance measures are reported in tables with rows and columns. Some municipalities and county governments, however, display important measures in eye-​catching charts. The Village of Pinehurst, North Carolina, posts online a performance dashboard that shows multiyear trend lines for key performance measures under such categories as “Safeguard the Community,” “Promote High Quality Development and Appearance,” “Protect the Environment,” and “Professionally Manage a High Performing Organization.”

39

40

the basics

One of the trend lines shows the fire department’s performance in meeting turnout time goals. Turnout time—​called reaction time on Pinehurst’s dashboard—​is the time it takes for a crew to exit the fire station following an emergency alert from the dispatcher. Pinehurst’s goal is for trucks to roll out of the station within 90 seconds at least 85 percent of the time. They met this goal in three of the four most recent years.

Figure 3.1  Emergency Turnout Time in Pinehurst Although printed here in black and white, performance is color-​ coded on Pinehurst’s online dashboard, with performance that meets or exceeds the goal earning green (at least 85 percent within 90 seconds in the case of turnout time), a yellow zone just below that, and a red zone designating unsatisfactory performance (79 percent or less within 90 seconds). A score, calibrated to 3.33 when barely entering the red zone and 6.67 when barely entering the green zone, is assigned to current performance. Accordingly, current performance in the yellow zone but just below the goal has earned a score of 6.14. Another trend line shows the average number of building inspections completed per day per inspector. Village management believes that competent and hard-​working inspectors can complete as many as 13 quality inspections in a day on average and encourages that level of efficiency. Management becomes alarmed if productivity falls well below that level, but it is also concerned if numbers regularly exceed that mark. A heavier workload could cause inspectors to rush through their work, perhaps sacrificing thoroughness and quality for the sake of speed. The gauge at the left in this graphic has two red zones: one on the left part of the dial for fewer than 8 inspections per day and one on the right for more than 13 inspections per day. Pinehurst’s inspectors have departed from the green zone and entered the red only once—​in 2018 when they averaged more than 15 inspections per day.

p erformance measur em ent and monitoring

Figure 3.2  Building Inspection Efficiency in Pinehurst Still another trend line shows the percentage of Pinehurst’s vehicles that are ready to go and available for use on a given day rather than down for maintenance or repairs. Good fleet maintenance operations expect to have only a small percentage of the fleet out of commission at any time. Pinehurst’s goal is a fleet availability rate of at least 98 percent, which it achieved in five of the last six years.

Figure 3.3  Fleet Availability in Pinehurst Source of graphics: Village of Pinehurst, North Carolina, Performance Dashboards. Accessed June 16, 2020 at ​www.vopnc.org/​our-​government/​performance-​dashboards/​. Used with permission. Many of the cities and counties that are especially adept at performance measurement have devoted considerable attention to the development of metrics that they can use not only to tell the government’s service delivery story to the governing body and external audiences but that will also provide important information to departments and programs. The measures will alert them when expected results are not occurring and, when corrective action is taken, will provide a means of monitoring the effectiveness of operating adjustments. Some have built their sets of performance measures by selecting metrics that have proven effective in other local governments and others have designed their own measures, sometimes using logic models to guide the design.

41

newgenrtpdf

42 the basics

Table 3.1  Reporting Performance Measures in Fort Lauderdale, Florida Water treatment and wastewater systems Department core process

Performance measures

Objective

FY 2018 actual

FY 2019 actual

FY 2020 target

FY 2020 projection

FY 2021 target

Operate and maintain a safe and efficient water and wastewater system

Number of failures per 100 miles of collection pipe (Collection System Integrity)

Minimize

8.7

4.7

2.0

4.0

2.0

Wastewater treated in million gallons per day (MGD) per full-​time employee (FTE)

Monitor

1.2

1.2

0.3

1.2

1.2

Potable water produced in million gallons per day (MGD) per full-​time employee (FTE)

Monitor

0.8

0.9

0.3

1.0

0.3

Compliance with primary drinking water standards (AWWA)

Maintain

100%

100%

100%

100%

Number of leaks per 100 miles of distribution pipe (Water Distribution System Integrity)

Minimize

47.2

43.9

21.7

22.7

21.7

Number of breaks per 100 miles of distribution pipe (Water Distribution System Integrity)

Minimize

10.5

11.2

19.4

16.6

19.4

99.0%

Source: City of Fort Lauderdale, Florida, FY 2021 Department Request: Public Works, 20. Accessed June 11, 2020 at ​www.fortlauderdale.gov/​home/​ showdocument?id=48203

newgenrtpdf

Table 3.2  Reporting Performance Measures in Coral Springs, Florida Development services Measure type

Responsible city government Growing local economy

Effectiveness

↑

Effectiveness

↑

Growing local economy

Effectiveness

↑

Growing local economy

Effectiveness

□

Growing local economy

Effectiveness

↑

Growing local economy

Effectiveness

↑

KPI

FY2017 target

FY2017 actual

FY2018 target

FY2018 actual

FY2019 target

FY2019 actual

FY2020 target

Department customer satisfaction rating Cycle time for small permits by the Zoning Division (Building Plan Review) (Days) Cycle time for sign permits by the Zoning Division (Building Plan Review) (Days) Cycle time for plan review (new and major/​minor) by the Zoning Division (Development Review Committee) (Days) Avg. number of days from receipt of the resident’s application for rehabilitation assistance to approval QTRAC data (30-​minute wait times) (New beginning FY2019)

95%

98%

95%

96%

95%

100%

95%

2

2.00

2

2.00

2

1.82

2

2

2

2

1

2

1.39

2

8

8

8

8

8

8.25

8

45

38

45

30

45

39.25

45

30

9.46

12

(continued)

p erformance me asurem e nt and monitoring

Goals

43

newgenrtpdf

44

Development services Goals

Measure type

KPI

Growing local economy

Effectiveness

↑

Growing local economy

Effectiveness

↑

Growing local economy

Effectiveness

↑

Premier community in South Florida

Efficiency

↑

Premier community in South Florida Premier community in South Florida Growing local economy

Effectiveness

↑

Effectiveness

↑

Effectiveness

↑

“Building” records requests within 10 business days (New beginning FY2019) Requested inspections completed within one day Percentage of plan reviews completed within 15 working days Timeliness ratio of CDBG spending: annual CDBG allocation available by July 31 Number of trees planted within the City Number of formal and informal neighborhood partnerships each year Percentage of code cases brought into voluntary compliance prior to administrative/​judicial process

FY2017 target

FY2017 actual

FY2018 target

FY2018 actual

FY2019 target

FY2019 actual

FY2020 target

90%

95%

90%

95%

100%

95%

100%

95%

99.95%

95%

90%

96%

90%

89%

90%

95%

90%

1.50

1.48

1.50

1.38

1.50

1.48

1.5

1,000

3,195

1,000

2,365

1,000

3,170

1,000

10

4

10

6

10

11

10

75%

80%

75%

81%

75%

79%

75%

the basics

Table 3.2 Cont.

Premier community in South Florida

Effectiveness

Premier community in South Florida

Effectiveness

84%

85%

84%

85%

85%

82%

85%

87%

95%

87%

94%

87%

94%

87%

85%

Note: CDBG = Community Development Block Grant; KPI = key performance indicator. Source: City of Coral Springs, Florida, Annual Budget: Fiscal Year 2020, p. 315. Accessed June 11, 2020 at ​www.coralsprings.org/​home/​showdocument?id= 15533

p erformance me asurem e nt and monitoring

Premier Effectiveness community in South Florida

 Percentage of respondents satisfied with City efforts at maintaining the quality of their neighborhoods (Resident Survey) ↑ Percentage of survey respondents satisfied with the City’s efforts to support quality neighborhoods (Biz Survey) Process business tax applications within 7 business days (New beginning FY2020)

45

46

the basics

BOX 3.2  LOGIC MODELS A logic model is a graphic depiction of the presumed relationships between a program’s resources, activities, outputs, and effects. It shows how each element leads to another. Purpose or mission of the program Inputs or resources

Activities

Outputs

Effects

Context or conditions affecting the work

Figure 3.4  Logic Model The model can be constructed in sequence from left to right beginning with inputs, but often it is better to build it in reverse—​that is, by beginning with the desired effects. Then consider what outputs would be needed to achieve these effects, what activities will produce these outputs, and what inputs will be needed to support the activities. In some models the effects are subdivided into short-​term and long-​term outcomes with the box for short-​term outcomes preceding in the logic model chain the box for long-​term outcomes. In this case the reverse approach would begin by first considering the desired long-​term effects and then considering the intermediate effects that should be expected to precede them. It is important to challenge each assumption in the logic model. Will these inputs be sufficient to support the planned activities? Are they the right inputs? Why should we expect these activities to produce the expected outputs? Would different activities be better? Can we actually anticipate the effects that we are predicting? Are there better means of achieving the desired outcomes? In other words, is a more logical model possible? Persons developing performance measures sometimes argue that having a logic model helps. The model clarifies what measures are most needed—​measures that will confirm that the prescribed elements are present and, most importantly, that the desired effects are occurring.

p erformance me asurem e nt and monitoring

Governments often normalize some of their measures—​that is, they adjust selected performance statistics to put them on a common scale that allows comparison to other governments and to an earlier version of themselves, if the government’s population is changing. For instance, rather than just reporting the number of crimes in the community, a government may report the crime rate per 100,000 population—​even if the community is much smaller than 100,000 people. A city of 50,000 population that experienced 125 aggravated assaults in a year would report a normalized statistic of 250 aggravated assaults per 100,000 inhabitants. By reporting their statistic per 100,000 population, they have adopted the scale used by the Federal Bureau of Investigation (FBI) and have made comparison to other communities much simpler. Normalized performance statistics are important for growing cities and counties, even when assessing their own performance over time. Many cities track the number of complaints in various departments and programs. If the city’s population is growing, the number of complaints is likely to increase, despite the program’s best efforts to keep the number down, simply because the opportunities for complaints are growing with added population. If the solid waste department in a growing community reports complaints not as a raw number but instead as “complaints per 1,000 households” or “complaints per 10,000 collections,” the department will be more likely to reveal the success of its customer service efforts, because this number—​the rate rather than the raw number—​might be going down.

Efficiency, quality, effectiveness, and service equity, too Usually it is pretty easy to keep track of the number of service units being produced, but these raw counts of output have less managerial value than more advanced measures of efficiency, service quality, effectiveness, and equity. Just knowing that output has remained constant over the past few years is of little comfort if the cost of each unit of output is increasing, the quality and effectiveness of the service is declining, or the service is being denied to some groups of eligible recipients. These are deficiencies that would be exposed by good measures of efficiency, quality, effectiveness, and equity. Good sets of performance measures will include not only output measures but some of these higher-​order measures as well. Together these

47

48

the basics

form a “family of measures” that collectively addresses different aspects of service and possible tradeoffs between them. Efficiency measures relate outputs to the resources required to produce them. Sometimes efficiency measures take the form of unit costs (for example, the cost per acre mowed or the cost per million gallons of water treated), but often they are expressed as outputs per labor-​hour, outputs per full-​time equivalent employee (FTE), or staff-​hours per 100 units of output. Measures of service quality can take a variety of forms. Often, they address responsiveness (for example, the percentage of potholes filled within two workdays), completeness or accuracy of the performance or the absence of errors (for example, the error rate in emergency communications or “missed stops” in refuse collection), and citizen satisfaction (for example, the percentage of citizens “satisfied” or “very satisfied” with recreation programs). Effectiveness measures reflect the extent to which performance objectives are being met. If a city or county has a target of collecting household refuse on the day scheduled at least 97 percent of the time, an effectiveness measure would report the percentage of time that on-​schedule collection actually occurs. Measures of service equity address the uniformity of service delivery across neighborhoods or other relevant categories of service recipients (for instance, race, gender, age, or income). This requires the disaggregation of performance statistics. A fire department might be proud of its citywide average emergency response time of 7 minutes, but this summary statistic of citywide responsiveness could hide consistent deficiencies in responsiveness to one or two neighborhoods that are revealed only when the metrics are disaggregated. Measures of service equity perform this function.

BOX 3.3  EVALUATING SERVICE EFFECTIVENESS FROM PERFORMANCE RECORDS: A FIRE INSPECTION EXAMPLE A fire inspection program that works as it is intended reduces the likelihood of fire in the inspected buildings. Recently inspected facilities should be less vulnerable than those inspected long ago. Although a computerized version of the system described below would no doubt be a welcome relief from the tedium of a manual process, the system’s logic is clear and the steps are illustrated in this simple example (see Table 3.3). Each month, the analyst enters the number of

newgenrtpdf

Table 3.3  Example of Estimating Fire Inspection Effectiveness Suppose there are only 10 buildings subject to inspection in the community.Their inspection histories for a year might look like the chart below. Building number

January

February

March

April

May

June

July

August

September

October

November

December

0 6 5 2 0 0 2 4 10 1

1 7 6 3 1 1 3 5 11 2

2 8 7 4 2 2 4 6 12 3

3 0 8 5 3 3 5 0 13 4

4 1 9 6 4 4 6 1 14 0

5 2 10 0 5 5 7 2 15 1

6 3 11 1 6 6 8 3 0 2

7 4 12 2 7 7 0 4 1 3

8 5 13 0 0 8 1 5 2 4

Number of months since last inspection 8 3 2 6 5 10 9 1 7 3

9 4 3 0 6 11 0 2 8 4

10 5 4 1 7 12 1 3 9 0

Notes: (1) A “0” entry means that the building was inspected in that month. Note that the inspection frequency varies considerably among the 10 buildings, and in some cases (such as Building 4) the inspection frequency varies for the same building. (2) A circled entry indicates that this building had a fire in that month. For example, Building 1 had a fire in February, and at that time Building 1 had not been inspected for nine months.

p erformance me asurem e nt and monitoring

1 2 3 4 5 6 7 8 9 10

49

newgenrtpdf

50 the basics

Table 3.3 Cont. Summary analysis Number of months since last inspection

Number of entries showing that this many months had elapsed since last inspection

Number of fires in buildings that had gone this many months since last inspection

Incidence of fire in buildings that had gone this many months since last inspection

0–​2 3–​5 6–​8 9–​11 12–​15 All entries

40 37 24 11 7 120

0 0 1 2 3 6

0/​40 = .00 0/​37 = .00 1/​24 = .05 2/​11 = .18 3/​7 = .43 6/​120 = .05

Source: Harry P. Hatry, Louis H. Blair, Donald M. Fisk, John M. Greiner, John R. Hall Jr., and Philip S. Schaenman, How Effective Are Your Community Services? Procedures for Measuring Their Quality, 2nd ed. (Washington, DC: Urban Institute and ICMA, 1992), 99. Reprinted with permission. Note: The fire incidence ratios shown in the last column indicate that a considerable increase in fire incidence occurs when buildings have gone without inspections for a long time. Of all fires, for example, 83 percent (five of the six) occur in buildings that have gone at least nine months without an inspection, but buildings are in that position only 15 percent of the time (18 of the 120 entries). In practice, the exact fire incidence likelihood numbers will not be known because data on all months will be collected for only a random sample of buildings in the program; however, the percentage increases in the ratios computed will be the same as the percentage increases in the underlying fire incidence values. Also in practice, some adjustments have to be made to reflect the possibility that, when a fire and an inspection in the same building occur in the same month, the fire may have preceded the inspection and thus not have been affected by it.

performance me asur eme nt and monitoring

months since the latest inspection for each of the major buildings in town. If a building was inspected that month, the entry for that building is 0. If a fire occurs in any of the buildings, a circle is drawn around the entry for that building, showing that a fire occurred “X” months following an inspection. If only large numbers are circled, the analyst may conclude that the inspections are effective—​that is, fire hazards in recently inspected buildings have been spotted, eliminated, and have not had time to develop anew. The analyst might question, however, the infrequency of inspections that produces numbers so large. If small numbers are circled, the analyst might question the quality or effectiveness of the inspections.

Scenario: Tracking the performance of work crews in Mineola, Missouri 1 “I just handled another inquiry from an out-​of-​state caller who heard about our system for tracking the performance of water system maintenance crews,” Peg Waugh announced to her boss, Sean Kobesky, director of Mineola’s water department. Peg is Kobesky’s administrative assistant and has received a steady stream of such calls. “This was an official from a city in New Jersey who was impressed with our tracking system and amazed by the motivating force it has on work crews. He said that their crews are not very task oriented and seem more interested in avoiding work than in completing it. I guess I am so used to this system that I would probably take it for granted without calls like that.” “It’s interesting that you should bring up our performance tracking system,” Sean replied. “I have just been thinking about it, myself. I would like to take some steps to strengthen it. Don’t get me wrong; I think it’s a good system, too, but I would like to make it even better if we can. I’d like for you to take a hard look at it over the next few days and see what suggestions you can make.” 1 Like other cities and counties depicted in scenarios in this volume, Mineola, Missouri, is a fictitious community. The performance tracking system in Mineola, however, is based on a system developed in the Canadian city of Winnipeg, Manitoba, as described in Stanley Y. Siu, “Performance Chart Increases Crew Productivity,” Journal of the American Water Works Association 84, no. 2 (1992).

51

52

the basics

Peg left Sean’s office thinking she had been given a futile assignment. “We have a good system now. I cannot imagine what we need to do to improve it,” she thought. “Why do we want to mess with something that has been so successful?”

Seeking to make a good measurement system even better Peg knew the history of Mineola’s performance tracking system well. She should, having recounted that history to so many callers! The system was established in 2014 and is based on a set of standard “work points” assigned to the common tasks performed by water system maintenance crews (see Table 3.4). A task that typically requires a large crew and lots of time to complete (that is, a high number of labor hours) is awarded more work points than a task requiring few workers and little time (that is, fewer labor hours). Some tasks are valued at four times the work-​point level of other tasks. Although some aspects of the point system are a bit subjective, work

Table 3.4  Maintenance Tasks and Work Points Water task number

Description

Work points

110 112 113 122 129 130 133 142 143 144 149 150 157

Excavated repair to service boxes Service excavation and repair (more than 2 inches) Service excavation and repair (2 inches or less) Service renewal (2 inches or less) Planned valve installation Excavated valve replacement All excavated valve box repairs Excavated hydrant repairs Planned hydrant installation Hydrant replacement Water main repair (pavement) Water main repair (grassy area) Water main mini-​renewals

25 70 62 82 80 84 53 62 100 100 84 69 *

Source: Based on the system used by the city of Winnipeg, Manitoba, as reported in Stanley Y. Siu, “Performance Chart Increases Crew Productivity,” published in Journal AWWA, vol. 84, no. 2 (February 1992), by the American Water Works Association. Adapted with permission. * Work points for task 157 are calculated by multiplying by 2 the length in feet of the renewed water main.

p erformance me asurem e nt and monitoring

records from prior years provided a reasonably solid basis for assigning points, and minor adjustments have been made from time to time. Each workday, the performance of each crew is plotted on a performance chart (see Figure 3.5). Although charts such as these look complicated, they are really quite simple. Each chart covers a two-​week period, and each column of the chart represents a single day. The person filling out the chart simply totals the work points for the tasks completed by the crew in a given day, divides the sum by the total labor hours available to the crew that day,

Figure 3.5  Crew Performance Chart Source: Based on the system used by the city of Winnipeg, Manitoba, as reported in Stanley Y. Siu, “Performance Chart Increases Crew Productivity,” published in Journal AWWA, vol. 84, no. 2 (February 1992), by the American Water Works Association. Adapted with permission.

53

54

the basics

and enters the result as “production units” at the bottom of the chart and as a point on the graph. If a crew earns 153 work points based on the tasks completed during a given day and has 32 work hours available to it that day (4 crew members times 8 hours), it is awarded 4.8 production units for that day (153 divided by 32). The system is so simple that Mineola officials estimate it requires only about two minutes per crew per day to calculate and record the daily results. Some in Mineola say that the beauty of the crew performance chart system is its simplicity. Others say it is the valuable management information the chart provides. Peg is convinced, however, that the key feature of the system is the feedback it provides to the crews themselves. These charts are rarely sent to the city manager or the city council, and on those rare occasions that a chart is sent, it is transmitted only in a summary form that combines the performance of all crews. The routine destination of the charts of individual crews is the wall of the crew room, right next to the performance charts for all the other crews. This way each crew can see its daily results, and everyone else can, too! Although most crews do not particularly care whether they are the top-​performing group, no crew likes to bring up the rear with the lowest number of production units. Having observed the comings and goings of water crews over the past few years, Peg has noticed an interesting reaction to this performance feedback. When a dip on the performance chart is attributable to bad weather, every crew’s performance drops and none reacts one way or another. On other occasions, however, a low-​performance day usually draws a reaction from the crew, often directed toward the work scheduler. Peg has observed more than one crew chief angrily confronting the work scheduler, saying, “What’s the idea of sending us from a job on one side of Mineola to a job on the other side and back again. You’re killing our performance with travel time and making us look bad!” Or, “If you are only going to give us 69 points of work to do, don’t send us out with such a big crew!” It is clear in such instances that a feedback system designed to appeal to the personal and professional pride of crew members is having an impact. Has the system improved performance overall? Peg has recited the favorable statistics repeatedly. Average productivity units for all district crews have risen steadily over the years (see Figure 3.6). When the water department examined the performance of selected tasks in more conventional

p erformance me asurem e nt and monitoring

Figure 3.6  Average Production Units of District Crews Note: Hypothetical results for the fictional city of Mineola, based on the actual experience of Winnipeg.

terms, it found that the number of work hours required for a given service or repair usually had declined (see Figure 3.7). “It’s a good system,” Peg thought. “Sean wants me to suggest improvements and I don’t want to return empty-​handed, but I’m stumped. What’s missing from this system?”

Key elements of an advanced performance measurement system A quick perusal of the literature on performance measurement did two things for Peg. First, it reassured her that the system in Mineola had far more pluses than minuses. Second, it helped her spot what appeared to be the system’s single deficiency, a weakness that could be remedied easily. It was clear that the crew performance system did not suffer from two of the most common shortcomings in local government performance measurement—​ failure to advance beyond collecting only workload

55

56

the basics

Figure 3.7  Average Labor Hours Required to Repair a Water Main Break Note: Hypothetical results for the fictional city of Mineola, based on the actual experience of Winnipeg.

measures and failure to use the measures for operating decisions. While many performance measurement systems rely exclusively on workload or output measures, the Mineola system did not simply count the number of repairs or tasks completed. Instead, it linked tasks with resources in a meaningful way to provide more sophisticated and more meaningful measures. Furthermore, the measures were being used in important ways in Mineola—​not the least of which was to provide meaningful feedback to employees on a daily basis. In many cities and counties, performance measurement is done annually, strictly to meet the demands of the budget office or the manager’s office rather than to serve the operating needs of the department. In Mineola, the measurement system’s primary use is at the department level. The deficiency in the crew performance chart system, Peg now realized, was that in stepping beyond raw workload counts it moved only in the direction of efficiency measurement rather than toward both efficiency and effectiveness measurement. The fundamental measures in Mineola’s system focused on efficiency—​that is, they related output (various maintenance

p erformance me asurem e nt and monitoring

tasks completed) to resources consumed in the form of work hours. The system ignores the quality or effectiveness of repairs. It also ignores the question of equity in how these services are delivered across neighborhoods. Without measures addressing at least the dimension of effectiveness, Peg cannot rule out the possibility that the quality of repairs might be declining along with work hours.

BOX 3.4  BUILDING PERFORMANCE MEASURES INTO MANAGEMENT SYSTEMS Local governments that are leaders in the use of performance measures tie their measures to other key management systems—​strategic planning and management, budgeting, human resource development, performance management, program evaluation, and others—​ for maximum effect. For example, performance measurement and reporting play a central role in the performance management cycle that consists of planning, budgeting, management, and evaluation.

Figure 3.8  The Performance Management Cycle Source: A Performance Management Framework for State and Local Government (Chicago, IL: National Performance Management Advisory Commission, 2010), 21. Used by permission.

57

58

the basics

Back in the director’s office “We can improve the performance chart system by adding some measures that address the quality and equity of the work being performed,” Peg blurted out as she walked through the doorway of Kobesky’s office. “Excellent suggestion!” “Perhaps we could do it by building quality and equity components into the production units,” Peg continued, “but I am inclined to keep it simple by merely supplementing the current system with a few measures of service quality or effectiveness, and then disaggregating these effectiveness measures by neighborhood to confirm equity in the delivery of services. I hope we discover that the quality of our work is improving or at least remaining constant, even as we improve efficiency. That is something we need to know.”

BOX 3.5  OTHER APPLICATIONS OF PERFORMANCE MEASUREMENT As demonstrated in this chapter, performance measurement can enhance accountability (for example, performance reports for department heads, managers, elected officials, and citizens) and can provide valuable feedback to supervisors and employees. Every department of local government can benefit in both ways. In addition, a well-​developed set of performance measures provides information regarding service quantity, efficiency, quality, effectiveness, and equity that can improve a local government’s planning and budgeting processes. Such measures are also fundamental elements of program evaluations and performance contracts. Furthermore, local governments that wish to improve operations by means of the process known as benchmarking will quickly discover that performance measurement is an essential ingredient in that endeavor as well.

References A Performance Management Framework for State and Local Government. Chicago, IL: National Performance Management Advisory Commission, 2010.

p erformance me asurem e nt and monitoring

Suggested for further information Ammons, David N. Performance Measurement for Managing Local Government: Getting It Right. Irvine, CA: Melvin & Leigh, 2020. —​—​—​. Municipal Benchmarks: Assessing Local Performance and Establishing Community Standards, 3rd ed. Armonk, NY: Routledge/​M.E. Sharpe, 2012. —​—​—​, ed. Leading Performance Management in Local Government. Washington, DC: ICMA, 2008. Hatry, Harry P. Transforming Performance Measurement for the 21st Century. Washington, DC: Urban Institute, July 2014. —​—​—​. Performance Measurement: Getting Results. 2nd ed. Washington, DC: Urban Institute, 2006. Hatry, Harry P., Donald M. Fisk, John R. Hall, Jr., Philip S. Schaenman, and Louise Snyder. How Effective Are Your Community Services? Procedures for Performance Measurement, 3rd ed. Washington, DC: International City/​ County Management Association and the Urban Institute, 2006. McLaughlin, John A., and Gretchen B. Jordan. “Using Logic Models.” In Handbook of Practical Program Evaluation, 4th ed., edited by Kathryn E. Newcomer, Harry P. Hatry, and Joseph S. Wholey, 62–​87. Hoboken, NJ: Jossey-​Bass, 2015. Poister, Theodore H. “Performance Measurement: Monitoring Program Outcomes.” In Handbook of Practical Program Evaluation, 4th ed., edited by Kathryn E. Newcomer, Harry P. Hatry, and Joseph S. Wholey, 108–​136. Hoboken, NJ: Jossey-​Bass, 2015. Poister, Theodore H., Maria P. Aristigueta, and Jeremy L. Hall. Managing and Measuring Performance in Public and Nonprofit Organizations: An Integrated Approach, 2nd ed. San Francisco: Jossey-​Bass, 2015.

59

4 SMOOTHING DATA TRENDS BY USING MOVING AVERAGES

Standing too close to a data trend line can sometimes lead an analyst to a hasty recommendation or a department head or manager to a hasty decision. Sometimes it is better to step back and take a broader view rather than overreacting to what might be an anomalous spike or trough in the data. When the occurrence of a single favorable or unfavorable event can seriously distort or misrepresent a department’s performance or a community’s condition, analysts and managers might find it easier to take a broader view by examining and reporting data as “moving averages” rather than analyzing data solely by narrower time periods.

Scenario: Zornig, Minnesota 1 Keri Kelly was dutifully working on the fire department’s performance measurement section for the upcoming budget. “This looks awful!” she 1 The various cities, counties, and local government officials depicted in the scenarios of this volume are fictitious.

smoothing data tr ends with moving ave rag e s

Table 4.1  Fire Losses in Zornig, by Fiscal Year Fiscal year 2016–​17 Fire loss Fire loss as a percentage of value of properties involved

2017–​18

$210,500 $262,300 5% 7%

2018–​19

2019–​20

2020–​2021

$212,387 5%

$338,258 2%

$1,088,600 30%

thought (Table 4.1), as she filled in the column for last year’s fire losses. Chief Harold Behr, known to his friends as “Smoky,” had submitted a figure that stood out like a sore thumb. “This can’t be right,” muttered Keri. “I’d better go see the chief.” Occasionally, departments are a little careless in completing their budget sheets. A column here or there does not add up; a subtotal is not calculated correctly; or a pair of digits in a performance measure are transposed. Budget analysts moan when they discover errors of that type, but privately they delight in discovering them. “Good morning, Chief,” said Keri as she strode into the fire department’s administrative office. After exchanging a few pleasantries, she got down to business. “Let me ask you about last year’s fire losses.” Fully anticipating that she had discovered a typo in the department’s material and that the chief would be puzzled by her inquiry, she was startled by his response. “It looks terrible, doesn’t it?” Chief Behr replied. “Do you remember the warehouse fire at the beginning of the year? It destroyed the structure and all its contents.” “Yeah, I do, now that you mention it. I had forgotten about that.” “We take pride in our fire suppression effectiveness,” the chief continued, “but that one was out of hand before we were even contacted. The building was totally engulfed when we arrived. We responded quickly, but we had no chance of saving that structure.” “So what do you want to do about this?” asked Keri. “Well, we are pushing for tougher regulations regarding sprinkler systems…,” the Chief began, as Keri cut him off.

61

62

the basics

“No, I mean what do you want to do about the performance measurement section of your budget page?” “What I would really like to do is drop that fire loss statistic out of the table,” replied Chief Behr, “but I guess we shouldn’t do that. It is unfortunate that one bad incident can so distort our record. This community has a better fire department than last year’s fire loss figure suggests.” “We will need to report it,” said Keri, “but, like you, I am concerned that this will leave an inaccurate perception regarding the quality of fire services that the citizens of Zornig receive from the city. At the very least, we will want to add an explanatory note to the table of performance measures, but let me give this some thought. I will get back to you soon.”

Moving averages In circumstances like the one facing the Zornig Fire Department, a truer depiction of reality may be achieved by examining performance or conditions over a broader slice of time than by focusing on only a single period. A recent, high-​profile occurrence can illustrate this point. In January of 2020 a worldwide coronavirus pandemic reached the United States. The illness struck with deadly force first in Washington state and New York before spreading quickly across the country. Governors and mayors issued stay-​at-​home orders one after another in hopes of slowing the spread of COVID-​19. By year’s end more than 345,000 Americans would die. The emotional and economic toll of the pandemic was great. As the nation dealt with the loss of life, it also faced a harsh blow to the economy. Businesses were closed and millions were unemployed. Governors and mayors anxiously monitored the numbers of new COVID-​ 19 cases, hospitalizations, and deaths, trying above all to avoid overwhelming their hospitals and also hoping for signals that they could begin reopening their economies. To reopen, they needed to “bend the curve” on new cases—​ marking a change from the early upward trajectory to a flattening or plateau and then to a downward slope on new cases. But the pattern of new cases from one day to the next was erratic. An increase in new cases one day could be followed by a sharp decrease the next, only to be followed by another increase. Should they react to the increase or to the decline? To get a clearer sense of what was actually occurring, the governors and mayors focused on the moving average—​typically, a five-​day or seven-​day moving average.

smoothing data tr ends with moving ave rag e s

35k

Confirmed new cases

30k 25k 20k 15k 10k 5k 0 Feb 2020

Mar 2020

Apr 2020

5-day moving average

May 2020

Actual data

Figure 4.1  Using the Five-​Day Moving Average to Monitor New Coronavirus Cases in the US Source: Dong E., H. Du, and L. Gardner. “An Interactive Web-​based Dashboard to Track COVID-​19 in Real Time.” Lancet Infectious Diseases 20, no. 5: 533–​534. doi: 10.1016/​S1473-​3099(20)30120-​ 1. Johns Hopkins University & Medicine, Coronavirus Resource Center. Accessed on May 25, 2020 at https://​coronavirus.jhu.edu/​data/​new-​cases. Used with permission.

Only a few new cases of COVID-​19 were reported through February and early March 2020, but by mid-​ March new cases rocketed upward (Figure 4.1). Day-​to-​day differences in new cases are depicted in ragged peaks and valleys of the underlying histogram shown in the figure. The smoother line graph is the five-​day moving average of new cases. New York was the state hardest hit in the pandemic’s early stages, but several other states also had more than their share of cases. Figure 4.2 shows new daily cases in the state of Washington, depicted as a seven-​day moving average. Once again, the line graph of the moving average smooths the more erratic pattern of new daily cases. In Zornig’s case, rather than reporting annual fire loss statistics year by year, the department might consider reporting the average annual fire loss for a three-​year or five-​year period. Using the average over a longer span of time has the effect of smoothing the data and blunting the impact of an unusually bad or unusually good year. The degree of shock that Chief Behr and budget analyst Kelly expected to witness on the faces of startled members of the city council when they gazed at last year’s fire loss figure

63

64

the basics

Figure 4.2  Using the Seven-​Day Moving Average to Monitor New Coronavirus Cases in Washington State Source: Washington State Department of Health. COVID-​19 Data Dashboard. Posted at www.doh. wa.gov/​Emergencies/​COVID19/​DataDashboard. Accessed on October 9, 2020.

would appear only if a recurring pattern of fire losses developed—​that is, only if the community experienced two or three bad years in a row. The effect of a single anomaly would be muted. Moving averages may be calculated by simply adding together the figures for the desired span of periods and dividing the total by the number of periods. For example, a three-​year moving annual average is calculated by adding three annual totals together and then dividing by three. A six-​month moving monthly average is calculated by adding six monthly totals together and then dividing by six. The formula for calculating a moving average is: Moving average =

x1 + x2 +... x n n

where x = the total for a single period n = the number of periods included in the moving average Zornig’s raw fire loss statistics for the last five fiscal years are reported in Table 4.1. However, only three columns of performance measures are reported in the budget. Table 4.2 shows how these columns would appear as single-​year statistics (in the top half of Table 4.2) and as moving averages (in the bottom half of the table). The moving average is not as sensitive to

smoothing data tr ends with moving av e rage s

Table 4.2  Depicting Zornig’s Fire Loss Statistics Fiscal year

Single-​period format Fire loss Fire loss as a percentage of value of properties involved

Three-​year moving annual averages Fire loss, three-​year annual average Fire loss as a percentage of value of properties involved, three-​year annual average (unweighted)

2018–​19

2019–​20

2020–​21

$212,387 5%

$338,258 2%

$1,088,600 30%

2017–​2019

2018–​2020

2019–​2021

$228,396 5.7%

$270,982 4.7%

$546,415 12.3%

data fluctuations in a single period, which gives it some advantages in more appropriately depicting performance or conditions over the long haul.

Postscript When Keri returned to Chief Behr with a suggestion that he consider using a moving average to report fire loss statistics, the chief’s deliberation took only three seconds. “Let’s do it!” he quickly responded. “There’s some luck as well as skill in having a good year or a bad year on fire losses. We might not be quite as good as we look in a lucky year, and I know we are not as bad as we looked last year. Taking a larger slice of time seems to remove some of the luck factor from the equation.”

FROM THE ELECTRONIC TOOLKIT BOX 4.1  MOVING AVERAGES Calculating moving averages using Excel is simple. First, enter the data that are being averaged. (For directions on opening Excel and entering data refer to Box 1.1.) In the case of the town of Zornig, Keri Kelly would enter the year-​by-​year statistics for fire losses in a row (or column) of the spreadsheet. Then, she would calculate the three-​year averages by using the AVERAGE function and grouping the data by threes.

65

66

the basics

Suppose, for example, that the fire loss statistics for FY 2017 through FY 2021 were entered from cell A1 to E1. Then to calculate the first three-​ year moving average (FY 2017 through FY 2019), Kelly would enter “=AVERAGE(A1:C1)” in whichever cell she wishes to have that result appear. Next, the three-​year moving average for the FY 2018 through FY 2020 period is calculated by entering “=AVERAGE(B1:D1)” in the cell of her choice. The same procedure would follow for calculating the moving average for the next period. It is easy to alter the time period covered by a moving average using Excel because changes in the amount of data averaged can be made simply with the click of a mouse. To make alterations in the average formulas, simply highlight the cell with the formula in it and then move the mouse to the formula bar at the top of the screen. Click anywhere in the formula bar and type any changes that are needed using the arrow keys and the regular keypad. It is also possible to make changes to a formula within the spreadsheet by double-​clicking the cell with the formula in it. The formula will be displayed in the cell (instead of the formula’s output) and can be edited. Yet another approach to editing the formula is to select the cell and then the F2 function key at the top of the keyboard. The F2 function key opens the cell selected for editing. When this is done the actual cells that go into the function will be highlighted on the screen and can be moved and adjusted in size to make the moving average range match what is desired. If time series data are set up in a logical manner (for example, with entries for different time periods all in a single row across multiple columns), then once the moving average calculation is done for one set of entries the embedded formula can be easily copied and applied to other sets of entries. Simply click on the cell with the moving average result (in which the formula is embedded); then place the cursor in the bottom right corner of that cell. When a cross appears, left-​click and drag the mouse to one or more adjacent cells to carry the formula to those cells. If the relevant data are arrayed in a row, then drag the mouse to the right to align the moving average calculations in a row beneath the data row. (Note that if the data had been arranged vertically in a column rather than in a row, you would copy the formula downward after setting

smoothing data tr ends with moving ave rag e s

up the first moving average calculation.) If the time period of the moving average needs to be changed (for instance, from a 3-​year to a 5-​year moving average), just alter the formula for the first calculation and then again copy and drag that new formula across the relevant cells for the new moving average. For users who have graphed their data, a moving average can be automatically added to the chart. The user simply selects a given data series (typically a bar or line chart), right clicks on the mouse and selects “Add a Trendline.” One of the options that will appear is a moving average, which the user can set at whatever value is desired. In this way a moving average can be easily added to a graph without the user having to do the calculations themselves.

BOX 4.2  OTHER APPLICATIONS OF MOVING AVERAGES Like their hypothetical counterpart of Zornig, Minnesota, and their real counterparts monitoring the spread of the coronavirus in 2020, several state and local governments use moving averages to report performance, especially when performance is susceptible to sharp fluctuations that can be influenced only partially by government action. Moving averages have been used to report fire losses and infant mortality rates. Moving averages can also be useful in forecasting revenues (see Chapter 32). The effects of an especially strong or a particularly weak year for a given revenue source are softened in trend-​based projections using this technique.

Suggested for further information Hatry, Harry P. Performance Measurement: Getting Results, 2nd ed. Washington, DC: Urban Institute, 2006. See page 156. The Economist Numbers Guide: The Essentials of Business Numeracy, 6th ed. New York: Public Affairs, 2014. See pages 102–​104. Wang, Xiaohu. Financial Management in the Public Sector: Tools, Applications and Cases. Armonk, NY: M.E. Sharpe, 2006. See pages 4–​8.

67

68

the basics

Web resources StockCharts.com, “Moving Averages—​ Simple and Exponential” https://​ school.stockcharts.com/​ d oku.php?id=technical_ ​ i ndicators:moving_​ averages For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

5 SAMPLING FOR ANALYSIS

For many types of analysis, examining every relevant case would be impractical. Carefully examining a representative sample of the complete set is a more practical alternative. Sampling is a common analytic technique. The Nielsen people do not contact all television viewers. The Gallup and Roper polls rely on samples. The Dow-​Jones averages are based on a relatively small selection of stocks. The Associated Press polls that produce football and basketball rankings reflect the  views of the sportswriters who are polled. They do not represent the opinions of sports fans in general and may even differ somewhat from the views of most sportswriters. The purpose of a sample is to help an analyst or researcher draw conclusions about a larger group without examining every element in that group. The key to the usefulness of a sample is how well it represents the larger group. Interviewing a sample of people drawn from a conviction list of traffic offenders may produce an interesting set of opinions on the local law enforcement and justice systems. If drawn with care, that sample and the

70

the basics

views expressed might well represent the larger group of traffic offenders. It would be inappropriate, however, to misrepresent the findings as reflecting the views of the general citizenry or even of the people who have come into contact in one way or another with the local police and courts. The method used in selecting a sample can dramatically affect the representativeness of that sample and the appropriateness of inferences to a larger group based on analysis of sample data. The intended inference will determine whether it would be wise or unwise to draw a sample from a particular source. For instance, it would be wise to draw a sample from a conviction list if the purpose is to make inferences about that group; it would be unwise if the purpose is to make inferences beyond that group. For much the same reason, public hearings are often poor devices for making inferences about the general public because such hearings rarely draw a representative cross-​section of the citizenry. Instead, they tend to attract disproportionate numbers of local activists, persons most directly affected by the topic being discussed, and others with “axes to grind.”

Randomization In most cases, the ideal sampling method is random sampling—​but this method is not always feasible. For a truly random sample, all potential observations or interviewees in the relevant “universe” have an equal chance of being selected for the sample. That way the chances of a biased sample are reduced. A random sample of police incidents, for example, could be drawn by writing each incident number on a separate slip of paper, mixing the slips thoroughly, and blindly (that is, randomly) drawing the cases for examination. As a simple alternative, a random sample could be drawn using a random numbers table or selected using a computer-​ generated set of random numbers. Suppose, for example, that a sample of 400 cases is to be drawn from a total population of 6,450 cases. If the researcher is lucky, each of the cases already has been assigned a four-​digit case number; if not, the researcher could assign each a number from 0001 through 6450. Then, through the use of a computer-​generated list of random numbers between 0001 and 6450 (see Boxes 5.1 and 5.2), a random sample can be selected.

sampling for analysis

FROM THE ELECTRONIC TOOLKIT BOX 5.1  DRAWING RANDOM SAMPLES BY SPREADSHEET To draw a random sample from a “population” of numbers, first create a list of the numbers that constitutes the entire population. To do this, enter 1 to represent the first number in the population into the first cell and then enter a 2 for the second number in the next cell. (For directions on opening Excel and entering data refer to Box 1.1.) Click on the first cell, hold down the left mouse button, and then move the mouse to drag the curser down to highlight both of the first two cells. With the two cells highlighted, position the mouse on the lower, right-​hand corner of the highlighted block until a thin, black cross appears. When the cross appears, click with the left mouse button, hold, and then drag the mouse down to the desired number of data points in the list, including the largest number in the population set. Then, the numbers will appear in order. For example, if the desired pool of numbers extends from 0001 to 6450, enter “1” in cell A1 and “2” in cell A2. Then after dragging the mouse down to cell A6450, a list from 1 to 6450 will appear. Think of these as case numbers or account numbers—​one for each member of the population. (This column of case numbers could be inserted in a spreadsheet next to the names or other identifiers of all members of the population.) After creating the population list, Excel users will select the Data Analysis option from the Data menu. To access the Data Analysis option, the user must first activate the Analysis ToolPak for Excel. See Appendix F for instructions on activating the Analysis ToolPak. In the pop-​up window that will appear after selecting Data Analysis, select Sampling from the choices provided. In the Sampling pop-​up box, enter the range of the list created above in the Input box. In the example above, the user would enter “$A$1: $A$6450” as the range in the Input box. Still in the Sampling pop-​up box, the default for the Sampling Method is the Random choice. Leave this highlighted and enter the size of the needed random sample in the Number of Samples box. For example if you need a sample of 100 observations, you would enter 100 in the Number of Samples box. Click “OK” and a random sample of the desired size will appear on a new worksheet. These random numbers identify the cases for which records should be extracted for analysis. This is your random sample.

71

72

the basics

Another approach to randomizing your population is to add a random number next to each population member. Simply add a new column adjacent to the population numbers and type in the function =RAND() and hit enter. This will produce a random number between 0 and 1. Copy this down the population list and every entry will now have a random number. This number is recalculated every time a new data entry is made anywhere on the spreadsheet or whenever the sheet recalculates. The user should highlight the range of RAND function numbers, copy the set, and then paste the result on top of the original functions cells using Paste Special Values. This will convert the constantly changing random numbers into a set of fixed but still random numbers. The list of data can then simply be sorted by this new column of random numbers (high to low or low to high). This effectively reorders the entire data set into a new random order. This approach has the benefit of letting you have the complete population in random order. Then, simply work down the list until the desired sample size is reached.

FROM THE ELECTRONIC TOOLKIT BOX 5.2  CREATING A RANDOM NUMBERS LIST FROM WEBSITES Lists of random numbers may be generated by computer. This can be especially helpful to researchers conducting telephone surveys who are intent on eliminating the threat unlisted numbers pose to true randomization in more manual systems. See, for example, Research Randomizer (www.randomizer.org), a website that generates random numbers in the range specified by the researcher. Other randomization websites include Random.org (www.random.org/​integers/​) and Graph Pad (www. graphpad.com/​quickcalcs/​randomN1.cfm). If the researcher wishes to draw 50 numbers from a particular telephone exchange (that is, a telephone “community” defined by the first three digits in the seven-​digit telephone number) within a city, county, or region, the specified range can run from the smallest possible number beginning with those three digits (that is, XXX0000) and end with the largest possible number (that is, XXX9999). The same procedure would be followed for each telephone exchange in proportion to the number of units in the various exchanges.

sampling for analysis

If the survey involves only residents, the researcher may choose to specify more random numbers than are actually needed, retaining the first calls completed to the desired number of residents in the telephone exchange and dropping numbers found to be unassigned or assigned to businesses. Some software programs will not only generate random telephone numbers but also will perform a variety of other survey-​ processing functions. These more elaborate programs are referred to as Computer-​ Assisted Telephone Interviewing (CATI) systems. See, for example, StatPac for Windows survey and analysis software (www.statpac.com); the Computer-​Assisted Survey Execution System (CASES), developed by the University of California at Berkeley (https://​cases.berkeley.edu/​); and the Fully Integrated Control System (FICS), developed by the Research Triangle Institute (www.rti.org).

Systematic and stratified samples When random sampling is impractical, systematic sampling and stratified sampling are among the reasonable alternatives. Systematic (interval) sampling is very common—​probably because it is simple and can be done more quickly than most other acceptable alternatives. A systematic sample is drawn by taking each case following a predetermined interval (for example, every 10th case). The appropriate sampling interval may be calculated simply by dividing the total population by the desired sample size. If the total population is 1,000 and the desired sample size is 200, the sampling interval would be 5 (based on the simple calculation: 1,000 divided by 200 = 5). The starting point would be selected randomly from the first five cases in a list of the total population. Thereafter, every fifth case would be added to the sample. The use of systematic sampling represents a trade-​off: although it is often easier and more practical to draw a systematic sample than a true random sample, a systematic sample is likely to be a somewhat less accurate representation of the population from which it was drawn. Stratified sampling requires some knowledge of important population characteristics—​ for example, percentage of African Americans or Latinos in a predominantly white community or, perhaps, percentage of

73

74

the basics

management employees in a workforce that is predominantly laborers. If the researcher is worried that the sampling process may overrepresent or underrepresent a minority stratum (for example, African Americans or Latinos in a community survey, management in an employee survey), the technique of stratified sampling can be used to ensure appropriate representation. One approach is to identify relevant strata within the total population and randomly sample each stratum, drawing a sample size that is proportionate to the relationship between the stratum population and the total population. Another approach would involve drawing fairly large samples from each stratum, the calculation of stratum statistics, and their consolidation on a weighted basis.1 Random samples, systematic samples, and stratified samples are common sampling techniques that produce useful research results. Audit procedures normally incorporate one or more of these methods in the sampling of transactions. Another prime example is the use of sampling techniques for properly conducted citizen surveys.

AVOID THIS COMMON ERROR BOX 5.3  “RANDOM” DOES NOT MEAN HAPHAZARD When government officials report “random” opinions based on a sidewalk poll or the calls they received last week, they use the word “random” in error. In a truly random sample, every member of the relevant population has an equal chance of being selected. This does not happen without a great deal of planning and care to ensure true randomness.

1 If, for example, a particular stratum represents only 2 percent of the total population, normal sampling might result in only 10 or 12 members of that stratum being selected—​too few to permit meaningful inferences. Therefore, the analyst might choose to draw a larger sample of representatives from this stratum to permit more reliable analysis. However, when these observations are combined with those from other elements of the total population, each stratum’s influence should be weighted according to its proper proportion of the whole, so as not to be overrepresented and to thereby distort the general findings.

sampling for analysis

Unfortunately, it is as easy to cite poor examples of sampling for the analysis of local government problems as it is to cite good examples. Local officials who base judgments on “public opinion” as expressed in a mail-​in newspaper poll or the “overwhelming 2-​to-​1 margin of support” among the 12 people who called their council members or the city manager to voice their opinions on an issue are mistakenly bestowing representativeness on an unrepresentative sample. Local officials should not assume that a technique is reliable simply because they have seen it applied elsewhere. For example, phone-​in and Internet polls regularly conducted on television sports programs are not necessarily representative of the total public, all sports fans, or even all sports fans who happen to be watching that program. They represent only that portion of the viewing audience that wants to voice opinions, has access to a telephone or computer, and is willing to make the effort to respond.

Sample size and confidence level The size of the sample is another important factor to be considered. Although a large sample is generally preferable to a small one, randomly drawn samples need not be huge.2 For example, an audit of invoices might sample 300 invoices if the total population of invoices is fewer than 1,000 or 400 invoices if the population of invoices is 30,000 (Table 5.1). Sample sizes for citywide citizen surveys commonly range from 300 to 700 households even in fairly large cities.3 A sample is expected to approximate the population—​the more accurate the approximation, the better. The only way to get absolute precision is to survey the entire population, but that is rarely practical. Surveying a sample

2 Increasing the size of a sample tends to increase its accuracy in representing the population from which it was drawn; however, the accuracy increases with the square root of the sample size. In other words, a sample must be quadrupled to double its accuracy. See Russell Langley, Practical Statistics Simply Explained (New York: Dover Publications, 1970), 45. For an online sample size calculator, go to www.raosoft.com/​samplesize. html. 3 Harry P. Hatry, Louis H. Blair, Donald M. Fisk, John M. Greiner, John R. Hall Jr., and Philip S. Schaenman, How Effective Are Your Community Services? Procedures for Measuring Their Quality, 2nd ed. (Washington, DC: Urban Institute and International City/​County Management Association, 1992), 173–​184.

75

76

the basics

Table 5.1  Prescribed Sample Sizes for an Audit of Invoices Total number of invoices

Recommended invoice sample size

Fewer than 1,000 1,000–​1,999 2,000–​2,999 3,000–​9,999 10,000–​49,999

300 335 350 385 400

Source: Charles K. Coe, Maximizing Revenue, Minimizing Expenditure for Local Governments (Athens: Institute of Government, University of Georgia, 1981), 20–​21. Used with permission.

is a more practical alternative, but settling for a sample instead of the whole introduces a degree of imprecision that is expressed in terms such as margin of error, confidence interval, and level of confidence. When a survey showing that 61 percent of respondents are satisfied with police services has a margin of error of ± 5 percent, the pollster is declaring that the corresponding percentage for the population as a whole is projected to be somewhere between 56 and 66 percent. This range is the confidence interval. Table 5.2 shows the relationship between sample size and levels of precision for citizen surveys. Level of confidence refers to the pollster’s degree of certainty that the sample reflects the parameters of the full population. A confidence level of 95 percent, as reflected in Table 5.2, asserts that drawing random samples of a given size would reflect population parameters to the specified degree 95 out of 100 times. The pollster might declare, for example, 95 percent confidence that the opposition to a bond proposal is between 25.4 and 34.6 percent, based on negative responses from 30 percent of a 400-​person sample. (This result is shown at the intersection of the “400” column and “30%” row of Table 5.2.) Formulas showing the relationship between sample size, confidence levels, and confidence intervals more precisely may be found in many statistics and survey research texts.4

4 For example, see David H. Folz, Survey Research for Public Administration (Thousand Oaks, CA: Sage Publications, 1996), 46–​54.

newgenrtpdf

Table 5.2  Relationship between Sample Size and Precision in a Simple Random Sample Poll: 95 Percent Confidence Level If the percentage of respondents giving the same answer to a question is:

…and the sample size is 50

100

200

300

400

1,000

…then there is a 95 out of 100 chance that the full population would respond to the same question at a percentage in the range specified below. 10% 20% 30% 40% 50% 60% 70% 80% 90%

0.7–​19.3 7.9–​32.1 16.3–​43.7 25.4–​54.6 35.1–​64.9 45.4–​74.6 56.3–​83.7 67.9–​92.1 80.7–​99.3

3.6–​16.4 11.7–​28.3 20.5–​39.5 29.9–​50.1 39.7–​60.3 49.9–​70.1 60.5–​79.5 71.7–​88.3 83.6–​96.4

5.6–​14.4 14.2–​25.8 23.4–​36.6 33.0–​47.0 42.8–​57.2 53.0–​67.0 63.4–​76.6 74.2–​85.8 85.6–​94.4

6.4–​13.6 15.3–​24.7 24.6–​35.4 34.3–​45.7 44.2–​55.8 54.3–​65.7 64.6–​75.4 75.3–​84.7 86.4–​93.6

6.9–​13.1 16.0–​24.0 25.4–​34.6 35.1–​44.9 45.0–​55.0 55.1–​64.9 65.4–​74.6 76.0–​84.0 86.9–​93.1

8.1–​11.9 17.5–​22.5 27.1–​32.9 36.9–​43.1 46.9–​53.1 56.9–​63.1 67.1–​72.9 77.5–​82.5 88.1–​91.9

The formula for calculating these ranges is the margin of error for proportions (MOE) formula: p× 1− p



n

)+

1



2n 

where the z-​score is selected based on the confidence level desired (95% in this case for a z-​score of 1.96) p is the percentage of respondents answering in the same manner for the given question n is the size of the sample For details on this formula, see Joseph L. Fleiss, Bruce Levin, and Myunghee Cho Paik, Statistical Methods for Rates and Proportions, 3rd ed. (Hoboken, NJ: John Wiley & Sons, 2003), 29.

sampling for analysis

(



MOE = ±  z score ×

77

78

the basics

BOX 5.4  OTHER APPLICATIONS OF SAMPLING Carefully drawn samples can serve a variety of analytic needs. The most obvious application is the sampling done for citizen surveys conducted to gauge satisfaction with services or perceptions regarding local issues. Other applications include user surveys focusing on a sample of clients of a particular service, audit or evaluation studies drawing conclusions from a sample of transactions or cases, and job studies that rely on observations at random times rather than continuous observation. When the city auditor in Austin, Texas, wanted to check the conscientiousness and accuracy of contractual meter readers, the first step was drawing a sample of 70 meters from the city’s entire set of meter reading routes and addresses.1 Next, an auditor checked and recorded the reading of the meter for each sampled address the day before and the day after a regularly scheduled reading. When the official readings were compared to the range established by the auditor readings, 100 percent were found to be consistent with expectation (Figure 5.1).

Figure 5.1  Meter Read Testing Process and Result Source: City of Austin, Water Meter Reading and Billing Accuracy (Austin, TX: City of Austin/​Office of the City Auditor, March 2018), 4. 1 City of Austin, Water Meter Reading and Billing Accuracy (Austin, TX: City of Austin/​Office of the City Auditor, March 2018).

sampling for analysis

References Folz, David H. Survey Research for Public Administration. Thousand Oaks, CA: Sage Publications, 1996. Hatry, Harry P., Donald M. Fisk, John R. Hall Jr., Philip S. Schaenman, and Louise Snyder. How Effective Are Your Community Services? Procedures for Performance Measurement, 3rd ed. Washington, DC: International City/​ County Management Association and The Urban Institute, 2006. Hatry, Harry P., John E. Marcotte, Therese van Houten, and Carol H. Weiss. Customer Surveys for Agency Managers. Washington, DC: Urban Institute Press, 1998. Langley, Russell. Practical Statistics Simply Explained. New York: Dover Publications, 1970. Weiss, Carol H., and Harry Hatry. An Introduction to Sample Surveys for Government Managers. Washington, DC: Urban Institute, 1971.

Suggested for further information Eller, Warren S., Brian J. Gerber, and Scott E. Robinson. Public Administration Research Methods: Tools for Evaluation and Evidence-​Based Practice, 2nd ed. New York: Routledge, 2018. “Population Sampling,” Chapter 7, 138–​154. Malan, Roland M., James R. Fountain Jr., Donald S. Arrowsmith, and Robert L. Lockridge II. Performance Auditing in Local Government. Chicago: Government Finance Officers Association, 1984. Manza, Jeff, Fay Lomax Cook, and Benjamin I. Page, eds. Navigating Public Opinion: Polls, Policy, and the Future of American Democracy. New York: Oxford University Press, 2002. Raaum, Ronell B., and Stephen L. Morgan. Performance Auditing: A Measurement Approach. Altamonte Springs, FL: Institute of Internal Auditors, 2001. The Economist Numbers Guide: The Essentials of Business Numeracy, 6th ed. New York: Public Affairs, 2014. See “Sampling and Hypothesis Testing,” pages 124–​144.

79

6 BASIC STATISTICAL TESTING CORRELATION AND CHI-​SQUARE

When two variables are related to one another, they are said to be correlated. If a local government official finds that hotter summer temperatures are reliably accompanied not only by bigger crowds at the outdoor swimming pool but also by a greater incidence of crime in the community, awareness of these correlations may lead to supplemental staffing at the pool and special crime mitigation steps when weather forecasters predict record heat. Knowledge of correlations between factors relevant to local government issues and service delivery can influence management strategies of all kinds. For instance, analysts in King County, Washington, examined the relationship between the number of miles of use per year by automobiles and other light-​duty vehicles and cost per mile for operating those vehicles (Figure 6.1). They discovered that heavily used vehicles tended to cost less per mile to operate. Those being driven 7,200 miles or greater per year were, in general, achieving the cost-​per-​mile optimum. Accordingly, the county’s policy calls for users having a need for fewer miles to meet their need by a pooled-​car

basic statistical t esting

Figure 6.1  Scatterplot Depicting the Relationship between Miles Driven and the Cost Per Mile for Owning and Operating Light-​Duty Vehicles in King County, Washington Source: Larry Brubaker, Elise Garvey, Laina Poon, and David Dean, Light Duty Fleet: Costs and Emissions Could Be Reduced (Seattle, WA: King County Auditor’s Office, April 28, 2015), 3. arrangement or some means other than by making an individual vehicle purchase.1 The relationship between usage and cost is apparent in the scatterplot. A more precise representation of the correlation between two interval-​level variables, such as these, may be calculated in the form of a correlation coefficient, ranging from +1.0 to −1.0. The interval level of measurement is the most precise level. Precise measures of height, weight, distance, cost, and time are examples of interval level measures. The correlation coefficient may be calculated for interval-​level variables in any local government function and can serve a variety of management purposes. However, not all variables of interest can be measured at the interval level, so calculating a correlation coefficient is not always possible. Sometimes variables of interest to local government officials are categorical in nature. For example, the data may be divided into income categories of 1 King County, Washington, Light Duty Fleet: Costs and Emissions Could Be Reduced (Seattle, WA: King County Auditor’s Office, 2015).

81

82

the basics

high, medium, and low, without precise income information for each observation. Or perhaps the variable distinguishes racial groups, gender, religions, or neighborhoods. For categorical variables, the chi-​square is a test that permits a manager or analyst to determine whether one variable is significantly related to the other. In this chapter we will begin with the calculation of correlation coefficients. Then we will turn to the chi-​square test.

Scenario: Satterfield, Alabama Chet Warren was polite to the caller but was noticeably agitated when he hung up the telephone. A chief of police is supposed to be calm, collected, and under control—​not to mention thick-​skinned—​and by all accounts Warren is an excellent chief, but this call got to him. “Sometimes people say things because they think it is fashionable or funny to say them,” he muttered. “But say it enough and people start believing it’s true!” Chief Warren turned to Lester Scott, an administrative assistant in the police department who was pulling some material from the file cabinet in the corner of the chief’s office. “I know it’s just a cliché to most folks, but I am getting pretty tired of hearing, ‘There’s never a police officer around when you need one.’ The woman on the phone just now made some serious allegations and concluded her comments with that little declaration. I heard the same thing yesterday in a half-​joking manner, I think, at the Lion’s Club. I wish I had a better response.” “We can’t put an officer on every street corner,” said Lester. “What do they expect?” “They expect us to deploy our officers wisely,” responded the chief. “If we do, we ought to be able to document the soundness of our scheduling and assignment patterns. And if we don’t, I need to get the situation corrected.” “What do you have in mind?” “Run some correlations. Look at the pattern of calls for service—​by day of the week, time of day, and neighborhood or assignment zone. See how well that pattern correlates with the number of patrol officers on duty at those times and locations. Let’s get the facts and go from there.”

basic statistical t esting

Pertinent statistics To determine the extent to which one variable (for example, officers on duty) increases or decreases in relation to changes in another variable (for example, calls for service), a measure of association must be obtained. The choice in this case is the correlation coefficient r, which was introduced by Karl Pearson and often is called the Pearson product-​moment correlation. This coefficient ranges from +1.0 to −1.0, depending on the nature and strength of the relationship between the two variables. If Satterfield’s deployment strategies match the call-​for-​service demand patterns precisely, then the correlation coefficient will be +1.0 or very close to that level. This would be a strong positive correlation, meaning that as the number of calls for service increases, so does the number of police officers on duty. In contrast, if the correlation coefficient approaches −1.0, then there is a strong negative correlation, meaning that Satterfield’s on-​ duty strength tends to drop just when there is the greatest demand for service. If this turns out to be the case, then the local critics would be proven correct. If the correlation coefficient falls somewhere in the mid-​range near 0.0, this would signify a weak or nonexistent relationship between the two variables. As calls for service increase, the number of on-​duty officers either is unaffected or it varies in an unpredictable pattern, sometimes increasing and sometimes declining. Chief Warren is hoping to find a strong, positive correlation between calls for service and officers on duty. Calculation of the correlation coefficient by hand is not especially difficult, but it is time-​ consuming and tedious. The wide availability of computer software that includes capability for calculating correlation coefficients—​even from spreadsheets—​makes hand calculation an increasingly less popular choice.2 Still, it is simple, as shown below. The formula for the correlation coefficient r is as follows: r=

N ∑ XY − (∑ X )(∑ Y ) [ N ∑ X 2 − (∑ X )2 ][ N ∑ Y 2 − (∑ Y )2 ]

2 See, for example, Brian C. Cronk, How to Use SPSS: A Step-​by-​Step Guide to Analysis and Interpretation, 11th ed. (New York: Routledge, 2020). Some popular spreadsheet programs, such as Microsoft Excel, possess the capability of calculating correlation coefficients. The coefficients displayed in Table 6.3 were calculated using Excel.

83

84

th e basics

where X is one of the two variables (for example, calls for service) Y is the other variable (for example, officers on duty) N is the number of sets of variables Σ signifies the summation (that is, addition) of the indicated variables

FROM THE ELECTRONIC TOOLKIT BOX 6.1  FINDING THE CORRELATION COEFFICIENT Calculating the correlation coefficient by hand can be a long process and leaves plenty of opportunities for careless mistakes that may affect the results. Excel offers a quick and reliable alternative to calculation by hand—​the CORREL function. First, enter the two sets of data. (For directions on opening Excel and entering data refer to Box 1.1.) Then, in any cell, enter the CORREL command followed by the cell names in parentheses for the two ranges of data. For example, if Lester entered the calls for service from Table 6.1 in cell A1 through cell A9 and the number of officers on duty in cell B1 through B9, then he would enter the formula “=CORREL(A1:A9,B1:B9)” to determine the correlation of .866 between the two sets of data. Excel users can also go to the Formulas menu and select the CORREL function or click on the fx symbol on the formula toolbar and select CORREL from the list of Excel functions. Adding functions at the fx symbol has the advantage that a pop-​up window for the function will show up and can be used to make sure the user enters the data correctly. This is true of all Excel functions. If the user wishes to find correlations for several variables, the Data Analysis option in the Data menu offers a good alternative. To make the Data Analysis menu available, the user must first activate the Analysis ToolPak in Excel. See instructions in Appendix F for how to do this. After selecting the Data Analysis option, a pop-​up box will allow the user to pick the Correlation tool. By specifying the block of data to be analyzed, the user can construct a complete correlation matrix comparing all of the data items in that block.

basic statistical te sting

In addition to Microsoft Excel, users may refer to several online correlation calculators, including: Pearson Correlation Software (www. wessa.net/​corr.wasp), Vassar Faculty website (http://​psy1.clarion.edu/​ mm/​studyRDE/​c06WebStuff/​co6_​resources/​statcalculators.html), and EasyCalculation.com (www.easycalculation.com/​statistics/​correlation. php).

When Lester began compiling information, he started with data for a recent Sunday, February 2. He separated information on calls for service and officer assignments according to the police department’s three response zones and further subdivided this information by the three daily work shifts. When he was through, Lester had nine pairs of data (n = 9), showing calls for service (variable x) and officers on duty (variable y) for each zone and shift combination (see Table 6.1). To calculate the correlation coefficient using the computational formula, Lester had to figure products and averages; he had to square some of the numbers and calculate square roots for others; and he had to do a good bit of adding and subtracting. But without too much trouble, he found the correlation coefficient for the two sets of variables to be +0.87 (see Table 6.2). He would be able to report to the chief that for Sunday, February 2, at least, there had been a strong, positive correlation between the patterns of service demand and officer deployment. Table 6.1  Service Calls and Officers on Duty, Satterfield Police Department, Sunday, February 2 Zone/​Shift I.

II.

III.

Total

A B C A B C A B C

Calls for service (X)

Officers on duty (Y)

2 4 2 3 5 2 3 7 2 30

1 2 1 1 2 1 1 2 1 12

85

86

the basics

Table 6.2  Calculating the Correlation Coefficient for Service Calls and Officers on Duty, Satterfield Police Department, Sunday, February 2 Zone/​Shift I.

Calls for service (X) A B C A B C A B C

2 4 2 II. 3 5 2 III. 3 7 2 Total 30 ΣX = 30    ΣY = 12 x̅ = 3.33    ȳ = 1.33 r=

=

Officers on duty (Y)

XY

1 2 1 1 2 1 1 2 1 12

2 8 2 3 10 2 3 14 2 46

N ∑ XY − (∑ X )(∑ Y ) 2

[ N ∑ X − (∑ X )2 ][ N ∑ Y 2 − (∑ Y )2 ]

(9 × 46) − (30 ×12)  (9 ×124) − 900 (9 ×18) − 144 414 − 360

=

, − 900) (162 − 144) (1116

=

(216 ×18)

54

=

54 54 = = 0.87 62.354 3,888

Getting a more complete picture Lester, of course, needed more than one day’s worth of statistics in order to tell whether the department’s patrol resources were being deployed wisely. He decided to gather information on calls for service and officers on duty, by response zone and duty shift, for a two-​week period. He could then examine correlations across zones and shifts for a given day, correlations across days for a given zone and shift, and the overall correlation across all zones, shifts, and days. Lester wanted to find if officers were being deployed imprudently in any zone or shift, or if the department was overstaffing a particular day. Lester found an overall correlation coefficient of +0.81 for the two-​ week period (see Table 6.3). To the extent that any demand-​to-​deployment

newgenrtpdf

Table 6.3  Officer Deployment-​to-​Demand Correlations, Satterfield Police Department, February 2–​15 Date

2/​2 Su 2/​3 M 2/​4 Tu 2/​5 W 2/​6 Th 2/​7 F 2/​8 Sa 2/​9 Su 2/​10 M 2/​11 Tu

2/​13 Th 2/​14 F 2/​15 Sa

Calls for service Officers on duty Calls for service Officers on duty Calls for service Officers on duty Calls for service Officers on duty Calls for service Officers on duty Calls for service Officers on duty Calls for service Officers on duty Calls for service Officers on duty Calls for service Officers on duty Calls for service Officers on duty Calls for service Officers on duty Calls for service Officers on duty Calls for service Officers on duty Calls for service Officers on duty Correlation

Zone II

Zone III

A

B

C

A

B

C

A

B

C

2 1 3 1 2 1 2 1 0 1 4 2 2 1 2 1 1 1 2 1 1 1 1 1 5 2 4 2 0.84

4 2 4 2 2 1 3 1 3 1 6 2 6 2 3 1 3 1 3 1 4 1 4 2 5 1 5 2 0.67

2 1 2 1 2 2 3 1 1 1 2 1 5 2 2 1 2 2 2 1 3 1 1 1 3 1 2 1 0.39

3 1 2 1 3 1 2 1 2 1 3 1 2 1 2 1 2 1 1 1 1 1 2 2 1 1 3 1 −0.03

5 2 2 1 3 1 3 1 5 2 6 2 6 2 3 1 3 1 3 2 1 1 3 1 7 2 7 2 0.82

2 1 3 1 2 1 1 1 1 1 5 2 3 1 1 1 2 1 1 1 0 1 1 1 2 1 6 2 0.86

3 1 3 1 0 1 3 1 1 1 5 1 4 2 2 1 1 1 2 1 2 1 2 1 3 1 4 1 0.32

7 2 5 2 2 1 5 2 4 1 9 3 8 3 5 2 3 1 4 1 6 2 3 1 9 3 8 3 0.95

2 1 3 1 3 1 1 1 2 1 5 2 4 2 1 1 2 1 0 1 3 1 2 1 5 2 7 2 0.84

Total

Correlation

30 12 27 11 19 10 23 10 19 10 45 16 40 16 21 10 19 10 18 10 21 10 19 11 40 14 46 16

0.87 0.85 −0.04 0.74 0.67 0.84 0.92 0.82 −0.05 0.31 0.73 0.48 0.88 0.85 0.81

Notes: (1) Duty shifts in Satterfield are A, 7 a.m. to 3 p.m.; B, 3 p.m. to 11 p.m.; and C, 11 p.m. to 7 a.m. (2) Correlations in the right-​hand column reflect the department’s sensitivity to differences in call-​for-​service patterns by time of day (that is, from one shift to another) and by location (that is, from one zone to another). (3) Correlations in the bottom row reflect the department’s sensitivity to differences in call-​for-​service patterns by day of the week.

basic statistical t esting

2/​12 W

Zone I

87

88

the basics

mismatches existed, they tended to be on weekdays, when calls for service and on-​duty staffing are usually lightest. In most cases of poor correlation, an extra officer had been assigned on a weekday to a shift or zone that had little need for reinforcement. On February 10, for example, an extra officer was assigned not to Zone III’s B shift, typically the heaviest duty assignment, but instead to the midnight shift in Zone I. Lester gathered his materials and headed to the chief’s office, hoping to catch him before his next appointment.

Chief Warren’s response Lester was in luck. The chief’s 1 p.m. meeting ended earlier than anticipated, so they had 20 minutes to review Lester’s table of correlation statistics before his 3 p.m. appointment. Chief Warren looked over the statistics on demand-​ and-​ deployment patterns and was delighted with what he saw. “This is terrific, Lester. I can’t say that I am happy with how we are handling everything, but overall it looks pretty good,” he said. “And where we have a few problems, I think we can develop some solutions.” “So you like this report okay, huh, Chief?” Lester asked, knowing already that he did, but hoping to get another compliment. “You bet, I do, Lester,” the chief responded. “In fact, I want one of these tables to be produced every two weeks. I want to receive a copy and I want other copies to go to everyone having responsibility for scheduling work shifts. We are going to make this part of our management information around here.”

AVOID THIS COMMON ERROR BOX 6.2  CORRELATION IS NOT CAUSATION Just because a correlation coefficient for a pair of variables is large, an analyst should not jump to the conclusion that one of the variables is causing the other. The correlation statistic establishes only that the two variables are related to each other. Perhaps one is causing increases or decreases in the other, or perhaps yet another variable is influencing

basic statistical te sting

changes in one or both of the variables being examined. In this chapter’s example, calls for service and officers on duty in a hypothetical community are shown to be related to each other, but neither directly caused the other. Declarations of causation—​that is, that changes in one variable caused changes in another—​based on simple correlation statistics are a common error of beginning analysts.

Later, Chief Warren met with his management team—​the department “brass,” as they were called. He described the correlation statistic as a tool for alerting them whenever they experienced a mismatch between workload and personnel. He distributed copies of Lester’s table and allowed Lester to explain the details. Lester loved the limelight. After 10 minutes, he was still going strong, when Chief Warren gracefully regained control of the meeting. “I want us to manage our resources wisely,” the chief said. “That means getting good results, while holding down costs. To do that, we’ve got to put our officers on duty when and where they are needed most. Get used to this table, because you are going to be seeing one like it every two weeks. I want us to consistently keep our overall correlation coefficient above 0.80, even if some dips on a given day or shift are inevitable.” “If what you’re getting at, Chief, is good planning,” interjected a sergeant, “how about using ‘officers assigned’ instead of ‘officers on duty.’ We know what the demand patterns are and we can draw up good duty rosters, but if someone is out on vacation or if someone is sick….” “No,” the chief interrupted, “what I am getting at is good management. That includes good planning, but it also means making adjustments when necessary. Shifting assignments, redeploying from light call zones to heavy call zones, occasionally calling in off-​duty officers for overtime—​these are things I want you, as managers, to consider, keeping performance and cost in mind, of course.” “The next time someone says ‘There’s never a police officer around when you need one,’ ” the chief continued, “I want to be able to prove them wrong. These correlation coefficients plus some good response time statistics should do the trick. If we are deploying officers wisely and they still think they don’t see enough officers around, then it probably means

89

90

the basics

we are understaffed, given their expectations—​not that our staff resources are being managed poorly—​and we will be happy to accept their help in getting more police officers.”

BOX 6.3  OTHER APPLICATIONS OF THE CORRELATION STATISTIC Good management often requires the development and implementation of strategies ensuring resource deployment patterns coinciding approximately with the peaks and valleys of service demand—​by time, location, or both (see Chapter 10 “Demand analysis”). A swimming pool needs more lifeguards on duty when conditions are crowded. A utilities office needs more clerks available for counter duty during hours when lines are most likely to form. Sometimes a simple graph showing patterns of service demand and staff on duty will be sufficient to convince the manager or analyst either that current strategies are adequate or that modifications are needed. Simply reviewing the graphs might be enough. In other cases, however, an appropriate statistic might be desired as well. A study of the San Diego Fire-​Rescue Department revealed remarkably consistent patterns of fire-​rescue incidents by time of the day over three recent years (Figure 6.2). If it had chosen to do so, the department could have gained a rough idea of how well the deployment of equipment and personnel matched these patterns by constructing a similar chart showing the number of personnel on duty or equipment in service, also by time of day, and comparing these charts. But using the correlation statistic might have been an even better, more precise option. By using actual data for on-​duty personnel and incidents, the department could have calculated the correlation statistic and determined the extent to which incidents and personnel deployment correlated with one another. The findings would challenge the wisdom of constant staffing—​that is, having the same number of personnel on duty around the clock. Most fire departments having full-​time personnel adopt a constant staffing pattern for firefighters assigned to fire pumpers and heavy fire equipment—​that is, those firefighters are on duty around the clock and off-​duty the next two days. Should the same strategy be employed for

newgenrtpdf

basic statistical t esting

Figure 6.2  San Diego’s Fire and Rescue Incidents by Time of Day for Three Recent Years Source: Citygate Associates, San Diego Fire-​Rescue Department Standards of Response Cover Review: Volume 2 of 3—​Technical Report. Folsom, CA: Citygate Associates, February 22, 2017, 46. Used with permission.

91

92

th e basics

personnel assigned emergency medical technician (EMT) duty? Most departments offering emergency medical services experience more medical emergency than fire emergency calls. Therefore, would it be wise to have EMTs working an 8-​hour shift, with fewer ambulances and EMTs on duty during the earliest hours of the morning and more on duty during the afternoons and evenings? The correlation statistic could help a department decide.

Chi-​s quare Chief Warren receives a monthly report showing the percentage of officers earning scores of “expert,” “pass,” and “fail” at the department’s shooting range. Recently he has noticed declines in the scores that have him concerned. When the chief mentioned the drop to one of his sergeants, the sergeant said he could not explain it but speculated that some of the female officers might be dragging the department down. Chief Warren called Lester into his office and told him about the sergeant’s comment. “I’d like you to do some analysis and see if you can figure out what is going on. Start with the sergeant’s speculation. See if there is anything to that. Then we will see if there are any other possible hypotheses we should explore.” Chi-​square is a statistic used in the analysis of categorical data. The chi-​ square statistic helps the analyst determine whether respondents in one category (for example, senior officers or men) are significantly different in a statistical sense from those in another category (for example, less-​ senior officers or women) with regard to another categorical variable (for example, scores of “expert,” “pass,” or “fail” at the shooting range). The chi-​square (depicted symbolically as X2) is calculated using the following formula: X2 =

∑(fo − fe )2 fe

where fo and fe refer, respectively, to the observed and expected frequencies for each cell in a table presenting one set of categorical data as rows and another set as columns.

basic statistical t esting

Table 6.4  Officer Proficiency at the Shooting Range Gender of officer

Shooting proficiency Expert

Female Male Total

6

Pass

11

Fail

Total

3

 7 13 20

The Satterfield police department has 20 officers altogether, as shown in Table 6.4.3 A recent push for greater gender balance brought the force to 7 female and 13 male officers. Last month’s shooting range scores were 6 expert, 11 pass, and 3 fail. These numbers are the totals for the rows and columns (known as “marginal”) of a contingency table showing officer gender and shooting proficiency. Because 35 percent of Satterfield’s officers are female, it might be expected that 35 percent of the spots in each of the three shooting categories would be occupied by female officers. For instance, if two of the six expert shooters were female, that would be 33 percent—​close to the 35 percent expectation. The expected frequencies for male and female officers can be calculated by focusing on the marginals and applying the proportions for rows to the totals for columns to calculate an expected frequency for each cell.4 If the proportion of women is 7 out of 20 officers (7/​20 or 0.35) and that proportion is applied to the “expert” column, the result is an expected frequency of 30 in the female-​expert cell (7/​20 x 6 = 2.1) (see Table 6.5). Now, suppose that the observed frequencies (the actual responses, as shown in Table 6.6) differ from the expected frequencies. The chi-​square statistic can be used to determine if the findings are statistically significant—​ that is, if they are sufficiently different from expected frequencies to conclude that differences in one variable (for example, differences in shooting proficiency) are likely to be related to differences in the other (for instance,

3 This exceeds the 12 who are on duty on any given day. Additional personnel are needed because individual officers are off-​duty two days each week and have a two-​week vacation each year. 4 Alternatively, the proportions for columns could be applied to the totals for rows to arrive at the same results.

93

94

the basics

Table 6.5  Calculating the Expected Frequency of Cells from Knowledge of the Marginals Gender of officer

Female Male Total

Shooting proficiency Expert

Pass

Fail

Total

fe = 2.1 fe = 3.9 6

fe = 3.85 fe = 7.15 11

fe = 1.05 fe = 1.95 3

 7 13 20

Note: fe is the expected frequency for a given cell based on the proportions found in the marginal. Calculation for a given cell is:

fe = For example, fe for upper left cell =

row marginal × column marginal N

7 × 6 = 2.1 20

Table 6.6  Actual Shooting Proficiency (“Observed Frequencies”) by Gender of Officers Gender of officer

Female Male Total

Shooting proficiency Expert

Pass

Fail

Total

4 2 6

2 9 11

1 2 3

7 13 20

gender). In the case of shooting proficiency, the calculations of the various elements in the formula yield a chi-​square of 4.011 (see Table 6.7). The value of chi-​square is said to be statistically significant only if it exceeds the relevant value shown in a chi-​square table (see Appendix A). Calculating the chi-​square yields one of the three things needed in order to use the chi-​square table. The second task is to decide how rigorous the test should be—​in other words, how small the probability (p) of a mistaken verdict should be (for example, a p of 0.05 imposes a more stringent test than a p of 0.10). A p of 0.10 will be used here. Finally, the degrees of freedom in the table of survey results must be determined. Degrees of freedom (df) can be calculated using the following formula: df = ( r − 1) ( c − 1)

basic statistical t esting

Table 6.7  Calculations for Chi-​Square Cell

fo

fe

fo − fe

(fo − fe)2

(fo − fe)2 /​fe

a b c d e f Total

4 2 1 2 9 2 20

2.1 3.85 1.05 3.9 7.15 1.95 20

1.9 −1.85 −0.05 −1.9 1.85 0.05

3.61 3.42 0.00 3.61 3.42 0.00

1.719 0.888 0.000 0.926 0.478 0.000 4.011

where r is the number of rows in the table and c is the number of columns. In the example, the table has two rows (female and male) and three columns (expert, pass, and fail), which yields 2 degrees of freedom.5 Armed with the chi-​square score and degrees of freedom (X2 = 4.011 and df = 2), the chi-​square table (Appendix A) can be used to determine that the results fall short of statistical significance at the 0.10 probability level. Only if the chi-​square were greater than or equal to 4.605 would it meet that level of significance. (Many academic studies use a more stringent probability level of 0.05, which would be met here only if the chi-​square were greater than or equal to 5.991.) Because we are not 90 percent confident that the two variables are related to each other (which is what the 0.10 probability level is telling us), we could not get our results published in an academic journal. But we might still have a finding strong enough to warrant some type of response in our own organization. If we look a little further in the chi-​square table, we will find that we reach the .20 level if the chi-​square is greater than or equal to 3.219. Our chi-​square of 4.011 meets that threshold. We know, then, that we can be more than 80 percent confident the two variables are related. Depending on the variables being examined and the sensitivity of the issue, this may be enough to warrant cautious steps to be taken in our organization.

5 If the marginals (row and column totals) and enough of the cell entries are known in a given table, the values for all the remaining cells in that table can be filled in. Degrees of freedom indicate how many cells have to be known in a given table, along with the marginals, in order to be able to fill out the rest of that table. In the case of a table with two rows and two columns, knowing just one cell value (df = 1) along with the row and column totals is all the information needed to complete the table.

95

96

the basics

Table 6.8  Shooting Proficiency by Gender of Officers: Raw Numbers and Percentages Gender of officer

Female Male Total

Number of officers Expert

Pass

Fail

Total

4 2 6

2 9 11

1 2 3

7 13 20

29% 69%

14% 15%

100% 100%

Percentages by gender Gender of officer Female 57% Male 15%

Lester prepared one more chart—​this one showing not only the numbers of male and female officers in each shooting proficiency category but also the percentages of each gender in the three categories (Table 6.8). Now he was ready to see the chief.

Sharing his findings Lester hurried to tell the chief what he had found. “I found almost the exact opposite of what the sergeant said might be the problem. Some of our female officers are among our best shooters,” Lester said. “The relationship between officer gender and shooting proficiency did not turn out to be statistically significant at the most stringent levels, but the probability was still reasonably high. Even more striking was that while the numbers are small, 57 percent of our female officers scored expert versus only 15 percent of our male officers. I’d say the sergeant was wrong and we need to get that message out there.” “I love it,” Chief Warren replied. “I will remember this for the next time someone suggests the female officers are dragging us down.” “Let’s check a couple of other things,” he continued. “Go back and take a look at officer seniority as one possible factor related to shooting proficiency and the frequency at which officers are going to the shooting range for practice as another. I would like to figure out what’s going on with these scores.” “I will get right on it,” Lester said.

basic statistical t esting

References Citygate Associates. San Diego Fire-​Rescue Department Standards of Response Cover Review: Volume 2 of 3—​ Technical Report. Folsom, CA: Citygate Associates, February 22, 2017. Cronk, Brian C. How to Use SPSS: A Step-​ by-​ Step Guide to Analysis and Interpretation, 11th ed. New York: Routledge, 2020. King County, Washington. Light Duty Fleet: Costs and Emissions Could Be Reduced. Seattle, WA: King County Auditor’s Office, 2015.

Suggested for further information Eller, Warren S., Brian J. Gerber, and Scott E. Robinson. Public Administration Research Methods: Tools for Evaluation and Evidence-​ Based Practice, 2nd ed. New York: Routledge, 2018. “Hypothesis Testing,” Chapter 15, 311–​316. Gibilisco, Stan. Statistics Demystified. New York: McGraw-​Hill, 2004. Meier, Kenneth J., Jeffrey L. Brudney, and John Bohte. Applied Statistics for Public and Nonprofit Administration, 9th ed. Stamford, CT: Cengage Learning, 2015. O’Sullivan, Elizabethann, Gary Rassel, Maureen Berner, and Jocelyn DeVance Taliaferro. Research Methods for Public Administrators, 6th ed. New York: Routledge, 2017.

Web resources College of Saint Benedict & Saint John’s University, “Tools for Science” www. physics.csbsju.edu/​stats/​contingency_​NROW_​NCOLUMN_​form.html EasyCalculation.com, “Correlation Coefficient Calculator” www.easycalculation. com/​statistics/​correlation.php Mathcracker, “Chi-​Square Test of Independence” https://​mathcracker.com/​ chi-​square-​test-​of-​independence Preacher, Kristopher J. “Calculation for the Chi-​Square Test” http://​quantpsy. org/​chisq/​chisq.htm Stangroom, Jeremy. “Chi-​Square Test Calculator” www.socscistatistics.com/​ tests/​chisquare2/​default2.aspx

97

98

the basics

Wessa, Patrick. Free Statistics Software, Office for Research Development and Education, version 1.2.1, 2020 www.wessa.net/​ www.wessa.net/​corr.wasp For a template and exercise associated with this chapter, see https://​tools fordecisionmaking.sog.unc.edu.

7 SENSITIVITY ANALYSIS

Many analytic techniques require an analyst to make a few assumptions or offer a prediction or two. Perhaps they will need to make assumptions about rates of compliance with a new regulation—​whether compliance will be consistent or vary across socioeconomic or age groups in the population. Or perhaps the analyst will need to predict fuel prices or inflation rates a few or several years into the future. Once the assumption or prediction is entered into the formula or model, calculations can be made and the analysis can be performed. But inevitably there is uncertainty in each assumption. What if the actual numbers turn out to be higher or lower? Would an erroneous assumption dramatically affect the analytic result or affect it just a little bit? In other words, how sensitive is the overall analysis to the assumptions embedded within it? The testing of an analysis’s vulnerability to errors in embedded assumptions is called sensitivity analysis. The objective of sensitivity analysis is to determine whether plausible differences in underlying assumptions in an analysis would substantially change the analytic finding and consequently the recommended course of action.

100

the basics

Scenario: Clay County, Virginia Donna Weldun was very busy getting ready for the upcoming Clay County Fall Festival. This was a major event for the Clay County parks and recreation department. As head of the department she had a lengthy list of things she needed to address in the few weeks remaining before the event. Having the annual festival run smoothly is a high priority for Donna and her staff not only because this is a cultural highlight for the community but also because it provides a major source of revenue for the department. Almost all of the parks and recreation staff members have significant roles in the festival, but they cannot come remotely close to handling everything themselves. In the past, the department had always been able to get enough adult volunteers to handle all the special assignments, but volunteering was down in the last few years. This year for the first time, someone on the planning team suggested enlisting student volunteers from the local high school. Because all high school students in Clay County have a community service requirement and an annual total of volunteer hours they must meet, this appeared to be a ready source of volunteers. Donna estimated that she needed about 25 student helpers to supplement the adult volunteers. She posted a request for 30 students, knowing that some might not show up, and was pleasantly surprised to see all 30 slots fill up quickly. But at today’s planning meeting, a staff member raised a concern about the student volunteers. “As a parent of a teenager,” said Jake Ramstead, “I want to raise a concern that you might not get half of those students to show up on the day of the event. I dearly love my son, but he’s not always on time or good about showing up.” Donna thought Jake’s comment might be a bit tough, as she found the high school kids to be responsible customers of many of the department’s offerings, but she didn’t have a lot of direct experience managing student volunteers. This was new territory and she was uneasy. “If 25 of the 30 student volunteers show up, we will be fine. But if it drops to 15, that is going to cause a disruption. And if it hits 10, we’ll be having serious problems.” “I don’t want to be overly alarming,” Jake said. “I just want us to avoid being caught off-​guard if we have some no-​shows.” “I suppose I could sign up some temporary workers or pay overtime to staff from other departments, but that’s money I’m trying to hold onto for

se nsitivity analysis

our long list of equipment needs. I’d spend it, though, if I thought we were dropping to fewer than 15 student volunteers, because I think this would cause noticeable disruptions and might even force us to cancel some activities. But I don’t want to spend the money if it isn’t necessary.”

Seeking additional advice Donna was uncertain about what to do. She decided to drop in on her friend Claire Dunstan, a management analyst in the county manager’s office who had been helpful in thinking through some issues in the past. Donna handed Claire a sheet of paper showing the staffing plan for the festival. It called for the use of parks and recreation staff, supplemented by adult and student volunteers. She explained that they needed 25 student volunteers but had enlisted 30 just to be sure to cover any no-​shows. “We have 30 signed up, but now I am worried about whether that will be sufficient. I have built in a little cushion, but is it enough? “So if I understand your concern correctly,” Claire said, “why not just expand the list of high school volunteers, say up to 50? Then, even if only half show up, you’ll still have your 25 needed volunteers.” “I don’t want to have a lot of people standing around with nothing to do,” Donna replied, “that could be a problem. And if we hope to get student volunteers for other parks department activities, we don’t want this to be a bad experience. Still, I really don’t want to come up short.” “I see,” said Claire. “I agree that we don’t want them to have a bad volunteer experience, if we can avoid it.” She reached for the sheet Donna had given her earlier. “You made an assumption right here about teenage volunteers actually showing up after saying they would. Now, you’re worried about your assumption. What you need to do is something called sensitivity analysis.” Claire explained sensitivity analysis and suggested that Donna check with someone having broad experience with teenage volunteers to get their thoughts on a reasonable range for predicting the percentage who will show up after signing up. “Then use those numbers in your calculation to get a better handle on what is likely. In other words, see how sensitive your staffing projection is to any error in your embedded assumption.”

101

102

the basics

Sensitivity analysis Sensitivity analysis is a simple method for testing how well an analysis would stand up to plausible changes in underlying assumptions. Would changing an underlying assumption to another realistic assumption dramatically change the findings? Would it change the findings enough to cause the analyst to reach a different conclusion and offer a different recommendation? Donna followed Claire’s advice and tracked down the teacher responsible for monitoring high school students’ volunteer hours in community service. She learned that records comparing sign-​ups to actual participation were spotty, especially for activities involving only one or two volunteers. For major events involving several student volunteers, records were better. Among these, the teacher reported that over the last two years one event scored 100 percent, with every volunteer showing up. The poorest record was for an event with only 67 percent of the sign-​ups actually participating. The teacher said that the average for all of these larger activities was 84 percent. Donna used the minimum, maximum, and average percentages from the high school’s recent experience to test the sensitivity of the staffing plan to her assumption regarding student volunteers—​the only movable part in the calculation. She quickly realized that her estimate that 30 sign-​ups would yield 25 volunteers was almost exactly in line with the high school’s average (Table 7.1). This made her feel pretty good about the estimate. If the percentage of sign-​ups actually showing up matched the record high, she would have 5 extra volunteers at the festival; but if it matched the record low, she would be 5 volunteers short. Importantly, the worst-​case scenario in the sensitivity analysis suggested that her previous fears of having only

Table 7.1  Sensitivity Analysis of Student Volunteer Yield Rate Assumption With 30 students signing up as volunteers, how many will actually show up for duty? Sign-​ups 30

Plausible yield rate ×

Maximum Expected Minimum

Showing up for duty 100% 83.3% 67%

=

30 25 20

se nsitivity analysis

half the student volunteers showing up were outside the plausible range and probably overblown.

Contingency planning With the sensitivity analysis results in hand, Donna gathered her staff for a brainstorming session with two objectives in mind: developing strategies to boost the odds of student volunteers showing up and developing a contingency plan in case they did not. Tom Frisk suggested, “We could send out a short email survey to the 30 students who have signed up. In the transmittal, we could thank them for being a volunteer, share some instructions, tell them how to get their volunteer hours recorded, and finally ask them to confirm they will be there for their assignment.” “Good idea,” Donna replied. “A follow-​up confirmation would probably increase their commitment. We might also want to do a second email just a couple of days before the festival as a reminder.” “Communication of this sort is kind of like checking the forecast before the weekend to know if you need rain gear or have to change plans,” Jake added. “On that note,” Donna said, “we probably should do some contingency planning. What if we end up on the low side with only 20 student volunteers? What do we do then? And if we have 5 extras, how should we use them?” By the time the meeting ended, they had outlined their contingency plan. If the email survey revealed any volunteer dropouts, they would increase sign-​ups enough to get back to 30. If for some reason they could not reach 30, Donna would consider setting up a contract for temporary workers or using employees from other departments on an overtime basis—​options she really hoped to avoid. If on the day of the event they had more than 25 student volunteers, the extras would be assigned to pre-​determined duties at one of the five activities attracting the greatest volume of festival goers at last year’s event. They would not be as busy as volunteers getting one of the regular assignments, but they still should have a good experience. If fewer than 25 student volunteers showed up for duty, they had a plan for that, too. If they were shorthanded by only one volunteer, the shortage

103

104

th e basics

would be absorbed by what was supposed to be a four-​member team handling one of the concession stands. The team at that stand could get by with three. If they were shorthanded by two or three volunteers, the contingency plan called for closing an activity that only been marginally popular at last year’s event. If they were short as many as five volunteers, a second activity was identified as the next to go. Hopefully they would not have to resort to this, but they had a plan for that contingency. As the meeting broke up, Donna felt a smile cross her face. Finally she was feeling more in control.

Sensitivity analysis with spreadsheets In the case presented in this chapter, Donna had only one assumption to test. A more complex case involving more sophisticated analysis might test multiple assumptions over a range of values. This kind of sensitivity analysis can be done easily with a spreadsheet. Even fairly simple modeling can be done to test different assumptions side by side. The beauty of a spreadsheet in these situations is that changing one or more assumptions can be done with little effort and without the need to recalculate a new set of numbers. Furthermore, Microsoft Excel provides several built-​in tools to make sensitivity testing even easier under the label of What-​If Analysis tools under the Data menu in Excel. See Box 7.1 for short descriptions of these tools and where to go to get more instruction.

FROM THE ELECTRONIC TOOLKIT BOX 7.1  “WHAT IF ANALYSIS” IN EXCEL SPREADSHEETS Spreadsheets are designed with the flexibility to change numbers easily and test the effects of different assumptions, making them an ideal tool for sensitivity analysis. Excel provides three tools that make sensitivity analysis even easier by eliminating any need for creating multiple copies of the same spreadsheet. These tools can be found under the “Data” submenu in Excel. In some older versions of Excel and on Mac computers, these tools may be found under “Tools.”

s e nsitivity analysis

1.

2.

3.

Goal Seek. Sometimes when making a calculation we would like to identify the value of a particular element in the calculation that will produce a certain result. This can be particularly useful as it might be used to calculate the switch point, which is where the decision would change. Rather than plugging in one number after another until you finally reach the desired answer, Goal Seek lets you set the value of some formula result to a number you choose, and then tells you how some other number value or assumption in your spreadsheet would need to change to produce the desired result. Data Tables. Many times we would like to test not one value for an assumption but multiple values. Data Tables lets you test one or two assumptions over a range of values that you specify. This spreadsheet tool provides you with a table showing the effect over a range of assumptions. For example, you might wish to examine assumptions involving six possible values for the inflation rate and seven possible values for population growth. Rather than testing the 42 possible combinations in 42 spreadsheets, Data Tables let you set up a single table which will show all 42 combinations in a simple summary. Scenario Manager. We may have several combinations of variables we want to test under different scenarios, perhaps ranging from best case to worst case. Scenario manager lets you set up a single spreadsheet to test one or more changing variables under different scenarios. These scenario results are saved in a manner that lets you easily move between them and even provides a summary of the differences. This avoids the need for multiple spreadsheets if you want to compare different scenarios where only the underlying assumptions change.

There are many sites on the Internet that provide instruction on these three tools in Excel. Three links are listed below. Alternatively, you may simply search for these techniques on a favorite Internet search engine using the terms, “What If Analysis with Excel,” “Goal Seek with Excel,” “Data Tables with Excel,” or “Scenario Manager with Excel.” Suggested for further information www.youtube.com/​watch?v=cP-​-​HZKdoGs https://​ s upport.office.com/ ​ e n- ​ u s/ ​ a rticle/ ​ i ntroduction- ​ t o- ​ w hat​if-​analysis-​22bffa5f-​e891-​4acc-​bf7a-​e4645c446fb4 www.excelarticles.com/​videos/​mrexcels_​learn_​excel_​794_​40_​what_​ if_​scenarios.html

105

106

the basics

Looking for sensitivity analysis opportunities Every analysis that relies on assumptions or predictions can benefit from sensitivity analysis. In the chapters ahead, try to spot the instances where testing one or more of the analytic assumptions might be an appropriate place for using sensitivity analysis. Assumptions about the cost of energy, number of staff members needed, future inflation, or how long vehicles might last all represent opportunities to employ sensitivity analysis to determine if the results of the analysis might change enough to lead to different recommendations. Sensitivity analysis allows us to gauge the vulnerability of our recommendations to uncertainties in the underlying analysis. It may also remind us of the importance of preparing a good contingency plan.

Suggested for further information Blom, Barry, and Salomon A. Guajardo. Revenue Analysis and Forecasting. Chicago: Government Finance Officers Association, 2001. See pp. 61–​67. Michel, R. Gregory. Decision Tools for Budgetary Analysis. Chicago: Government Finance Officers Association, 2001. See pp. 82–​87. Schroeder, Larry. “Forecasting Local Revenues and Expenditures.” In Local Budgeting, edited by Anwar Shah, 53–​77. Washington, DC: World Bank, 2007.

Web resources Georgia Government Finance Officers Association, “Sensitivity—​ How Confident Are You in Your Financial Projections?” https://​ggfoa.org/​press-​ releases/​sensitivity-​how-​confident-​are-​you-​in-​your-​financial-​projections New Zealand Treasury Department, “Approaches to Sensitivity Analysis” https:// ​ t reasury.govt.nz/ ​ i nformation- ​ a nd- ​ s ervices/ ​ s tate- ​ s ector-​ leadership/​investment-​management/​better- ​business- ​cases-​bbc/​bbc-​ methods-​and-​tools/​approaches-​sensitivity-​analysis US Department of Transportation, Federal Highway Administration, “Sensitivity Analysis” www.fhwa.dot.gov/​policy/​2015cpr/​pdfs/​chap10.pdf Wikipedia: The Free Encyclopedia. “Sensitivity Analysis.” Posted April 9, 2020 at https://​en.wikipedia.org/​wiki/​Sensitivity_​analysis For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

8 BREAK-​EVEN ANALYSIS

Sometimes initiatives are proposed that will cut costs or earn additional revenues once the initial expenses and related operating costs are covered. By figuring out the volume of activity needed to reach the break-​even point, an analyst can assess the initiative’s financial viability.

Scenario: Sweet Melody, Tennessee Several weeks ago, the city manager put everyone on notice that this would be a lean budget year. “I want every department looking for ways to save money,” Dotty Parton had said. “Leave no stone unturned. Think outside the box.” Clichés made management analyst Rita McIntyre wince and this double cliché had sent a double shudder down her spine. Still, Rita got the point. She had been working for the past several days with Artie Robbins, the superintendent of the city garage, to see whether they could find any savings in his operation. Artie was a conscientious team player who would have been diligent in pursuing savings anyway, but it didn’t

108

the basics

hurt when the city manager promised that most of the savings would stay home to help the department that produced those savings meet its own budget needs. One of the possibilities Artie and Rita spotted was intriguing, but it would require an investment in new equipment before the city could hope to save some money.1 Each year the city sent all of its vehicles to private garages for their annual state inspection. This year’s cost was $30 per vehicle. One of the city’s mechanics already was qualified to perform inspections, and the others could qualify without too much trouble. The city garage could be authorized to perform the inspections on its own vehicles if only it had the proper testing equipment, which Artie priced at $6,500. This was all it lacked to qualify. “This looks like it has some possibilities,” Rita said. “We will need to do a break-​even analysis to find out.”

Break-​e ven analysis Break-​even analysis in an easy-​to-​apply technique that reveals the volume of activity or use that would be required to recoup one’s investment in a revenue-​producing or cost-​saving project or piece of equipment. The break-​even formula is as follows: fixed cost = break-eeven number of users or units of use net revenue or net cost avoidance per use where net revenue or net cost = avoidance per usee

revenue or cost variable cost avoidance per use − per use

1 The Sweet Melody scenario, like all others in this volume, is fictional. It is based, however, on an analysis performed for the town of Carrboro, North Carolina, by a team of graduate students from the University of North Carolina at Chapel Hill. See Carrboro Fleet Maintenance: A Review and Analysis (Local Government Productivity Analysis Project, Master of Public Administration Program, University of North Carolina at Chapel Hill, May 30, 2006).

bre ak-e v en analysis

Fixed costs are expenses that are incurred whether there is one customer or many, one unit of service or hundreds of units—​in other words, regardless of the volume of activity. Variable costs, on the other hand, are expenses associated with the volume of activity. For example, some local governments invest in red-​light cameras to document instances of drivers running red lights and to produce citations for violators. Although these local governments may prefer to emphasize their traffic safety objectives, they might nevertheless wish to at least break even financially. In this instance, the costs of the camera equipment and installation are fixed costs that are unaffected by the number of red-​light runners; the costs associated with processing the images, issuing citations, and receiving payments are variable costs that are affected by the number of red-​light violations. In the case of Sweet Melody’s city garage, the cost of new testing equipment would be a fixed cost. If the city had to hire an additional mechanic to perform the state inspections, this too would be a fixed cost because once hired the mechanic would be paid regardless of the number of inspections performed. Fortunately for Sweet Melody, an additional hire will not be necessary. In fact, Rita and Artie concluded that there would be little, if any, labor component even in the variable costs. The Sweet Melody city garage already has an employee designated as an “equipment service worker I,” who performs a variety of duties including changing tires, sweeping up, and delivering vehicles; but on many occasions mechanics also are pressed into duty to deliver the vehicles needing work in private garages, such as state inspections. Time spent driving and waiting is time lost for mechanic work. Keeping the state inspections in-​house would reduce the need for mechanic drivers and would actually save about as much time as would be spent performing inspections. In essence, the net variable cost for labor will be zero, and the variable costs for any parts needed to pass inspection will be lower when purchased in-​house compared with the amount that would be added to the inspection fee by a private garage. Thus, the only variable costs would be for supplies. The break-​even analysis for the city garage’s proposed takeover of state inspections of city vehicles is shown below. The garage will break even if it performs 38 inspections per year.

109

110

the basics

The formula is fixed cost = break-even number of users ( or units of use ) net revenue (or net cost avoidance ) per use where net revenue(or net cost avoidance) = revenue (or cost avoidance)per use-variablecost per use per use This is how to apply the formula to the Sweet Melody proposal: netcost avoidance = $30 fee per state inspection − $2.30 in supplies per inspection ance per use performed by private garage = $27.70 net cost avoidance per inspection

Thus, fixed cost $1,050a = = 38 inspections needed in order to break even net cost avoidance per use $27.70 The annual fixed cost shown here is based on straight-​line depreciation of the testing equipment. The equipment’s cost is $6,500, and its projected life is six years, with salvage value of $200. See Chapter 18 for instructions on straight-​line depreciation. a

Back in Sweet Melody “This looks like a winner,” Artie declared. “We will easily surpass the break-​even point with the size of our fleet. We sent 64 vehicles out for state inspections last year. With this change, I think we will come out a little more than $1,270 ahead each year. That’s not a lot, but it will help a little with next year’s budget.”

bre ak-e v en analysis

“Don’t spend it too quickly, Artie,” Rita replied. “That break-​even point is based on annualized expenditures for the equipment you’ll need. Because you will be buying the equipment up front, we still need to figure out a financing plan or we will have to convince the city manager that this cost-​ saving idea is good enough that you can have some of the budgetary advantage now even though you won’t recoup all the costs this year.”

References Carrboro Fleet Maintenance: A Review and Analysis. Local Government Productivity Analysis Project. Chapel Hill, NC: Master of Public Administration Program/​University of North Carolina, May 30, 2006.

Suggested for further information Michel, R. Gregory. Decision Tools for Budgetary Analysis. Chicago: Government Finance Officers Association, 2001. See pp. 20–​23.

Web resources Corporate Finance Institute, “Break Even Analysis” https://​corporate financeinstitute.com/ ​ r esources/ ​ k nowledge/​ m odeling/​ b reak-​ e ven-​ analysis/​ Gallo, Amy, “A Quick Guide to Breakeven Analysis” https://​hbr.org/​2014/​07/​ a-​quick-​guide-​to-​breakeven-​analysis Square Capital, “Break-​Even Analysis 101” https://​squareup.com/​us/​en/​ townsquare/​how-​to-​calculate-​break-​even-​point-​analysis For a template and exercise associated with this chapter, see https://​tools fordecisionmaking.sog.unc.edu.

111

9 DETECTING MEANINGFUL DIFFERENCES IN AVERAGE PERFORMANCE

Local governments collect lots of information about the performance of government workers and crews—​calls received, applications processed, development plans reviewed, and street potholes repaired, for example. Often, this information is compiled in the form of averages—​the average number of calls received per workday; the average number of applications processed by the human resources (HR) department or by an individual HR technician per workday; the average number of development plans reviewed by the planning department or by an individual technician per workday; and the average number of potholes repaired per crew per workday. A supervisor reviewing these numbers can easily see whether the average performance of a crew or individual is higher or lower when compared to the averages of others or to their own average performance in the past, but are these differences really meaningful? Sometimes they are. And sometimes, not so much. As a manager, it is important not to make too big a deal about insignificant differences. Whether the difference between the average performance of one worker or crew and the average performance of another is significant or insignificant is not simply a matter of the importance of the task or of one number being

me aningful diffe r e nc es in p e rformance

a little bigger than the other. Although the mean performance is relevant in comparisons, so is the dispersion of all the various instances of performance. For example, two HR technicians might each process 12 to 18 employment applications per workday, each occasionally handling as few as 12 and as many as 18 applications. If one of the technicians averaged 14.2 applications per day during the last quarter and the other averaged 15.5, it might be a mistake to make too big a deal about the superior performance mark of the latter, depending on the dispersion pattern (Figure 9.1). After all, in the performance depicted here each had two or three equally strong, high-​performance days (processing 18 applications) and each had days with equally weak low-​ performance output (12 applications). The dispersion across the range of performance was large in both cases. But suppose that the dispersion of performance by these two technicians was tight rather than wide, even though each maintained the same average as before (14.2 and 15.5, respectively). Suppose that the first technician had no day in which he processed fewer than 14 applications and no day of processing more than 15 applications, while still averaging 14.2 applications per day (Figure 9.2). Dispersion in this case would be very small. And suppose that the second technician had no day in which she processed fewer than 15 applications and no day of processing more than 17 applications, while still averaging 15.5 applications per day. Again, the clustering around the performance mean is very tight. Here, the performance of the second technician is clearly superior to the performance of the first—​even though their averages have remained 14.2 and 15.5!

Number of applications processed in a day 11

12

13

14

15

16

17

18

19

Averages

Technician 1

14.2

Technician 2

15.5 Average

Figure 9.1  The Means Indicate a Performance Difference, But Dispersion Patterns Suggest It Is Not Very Big or Especially Praiseworthy

113

114

the basics

Number of applications processed in a day 11

12

13

14

15

16

17

18

19

Technician 1

14.2

Technician 2

15.5 Average

Figure 9.2  Dispersion Patterns Clarify the Superior Performance of Technician 2 Luckily, there is an easy way to analyze the difference in mean scores to see if the difference is statistically significant. A technique called the t-​test considers both the mean and the standard deviation of two sets of observations to determine if the difference in means is statistically significant. The same technique that can assess the significance of differences in mean performance of different individuals or crews can also be used to assess differences in delivery of services to various neighborhoods or segments of a city or county.

Scenario: Little Gotham, Connecticut The city of Little Gotham has long been proud of its status as an early adopter of 311, the special non-​emergency telephone number that the public can call for information about city services, to make a complaint, or to report graffiti, a pothole in the street, or some other problem. Citizens who might not know which department to call about a damaged stop sign, a graffiti-​ defaced wall, or a broken streetlight—​and might have been tempted to call 911, despite the non-​emergency nature of their concern—​can just dial 311 to get their problem addressed. The employees in the 311 customer contact center handle all such calls and relieve some of the burden that otherwise would fall on the 911 emergency dispatch center. The city of Little Gotham had always considered this a win-​win: better service to citizens and a good way to lighten the burden on the city’s emergency telecommunicators.

me aningful diffe r e nc es in p e rformance

Performance measures proudly displayed in the city’s annual report show that crews have been responding more quickly than in previous years to all manner of citizen concerns. Recently, some vocal residents of the Scarboro neighborhood of Little Gotham began to raise some troubling accusations about numbers buried beneath the surface of the city’s rosy reports. Scarboro is the city’s largest predominantly African-​American neighborhood and a few of its residents have accused the municipality of slower and poorer responses to their concerns. During the public comments time at the latest city council meeting, Harold Jeffords was especially outspoken on this matter, declaring, “The overall numbers might look good, but that doesn’t mean Scarboro is getting good services. Treat us like you treat Westchester!” Westchester is the neighborhood where the country club is located and many of the city’s most affluent residents reside. In the wake of Jeffords’s rebuke, three council members separately contacted city manager Jason Cole, asking if there was anything to these accusations. Cole promised to look into it. Back in the office, the city manager huddled with management analyst Mattie Simpson. “I want you to take a hard look at our service delivery patterns in different neighborhoods. See if we’re guilty of discriminating against the Scarboro neighborhood in any way. If we are, we need to know about it and we need to get it fixed. Use the data the 311 center collects on service requests and responses, and subdivide the data by neighborhood. Perhaps you’ll find the answer there.”

Evidence of discriminatory service? Mattie secured detailed service data for the past two years from the 311 center, just as the city manager had suggested, and divided the data by neighborhood and by types of service call. She found that responsiveness differed sharply from one type of service call to another—​the average response time to citizen requests for the removal of roadway hazards, for example, was 1.2 hours, while graffiti was removed in an average of 1.8 workdays from the time it was reported and responses to streetlight outages averaged 3.2 workdays. Differences from one neighborhood to another in average responsiveness for a given service type were not as large.

115

116

the basics

Mattie rank-​ordered the neighborhoods according to mean response times for each type of service call and found that no neighborhood had the most favorable average response time for every service type—​not even Westchester. However, she found that Westchester’s averages were never worse than the citywide mean for any service type and ranked among the top three neighborhoods for four services. In contrast, Scarboro’s rankings bounced around more from one service type to another. Scarboro’s mean response times were sometimes more favorable than the citywide average for a particular service and sometimes less favorable. In general, the average response times did not seem all that different from one neighborhood to another and in the rank-​ordering none of the neighborhoods came in last every time. Still, a suitable response to the residents of Scarboro would need to address the matter raised by Jeffords: Are the residents of Westchester being treated better than those of Scarboro? In Westchester-​to-​Scarboro comparisons, Westchester’s mean response times were more favorable than Scarboro’s for every service type except responsiveness to requests for smoke detector inspections.1 The differences never seemed glaring to Mattie, but the pattern worried her. One of the biggest responsiveness gaps was for streetlight outages, where responses in the Westchester neighborhood occurred in an average of 2.9 workdays and in Scarboro, 3.3 workdays (Table 9.1). To see whether the pattern was systemic and might be evidence of a pattern of discriminatory service or might be merely coincidental, Mattie subjected the data to a t-​test.

Testing the difference between two groups Two sets of observations drawn randomly from the same population are unlikely to be identical; but they are likely, if the sets are large enough and truly drawn randomly, to be fairly similar. If, instead, the two sets are remarkably different from each other, we might begin to suspect that they were not drawn from the same population. They might have come from distinctly different populations.

1 The city’s smoke detector program offers a free smoke detector to anyone living in a residence without one, but first an inspection is required so fire department personnel can advise the homeowner about the best location for the smoke detector.

me aningful diffe r e nc es in p e rformance

Table 9.1  Comparing Responsiveness to Streetlight Outages in the Westchester and Scarboro Neighborhoods Responsiveness to 311 calls regarding streetlight outages (time from call to response, measured in half-​workday increments)

Mean

Westchester

Scarboro

2.5 3 2 3 3 3.5 3 3.5 3 2.5 2.9 workdays

3.5 2.5 3 3 3.5 4.5 2.5 4 3 3.5 3.3 workdays

Let’s consider the theoretical possibility that every resident of Little Gotham could have a problem with a faulty streetlight and that the city’s actions to deal with those problems would reflect municipal responsiveness. If responsiveness is unaffected by neighborhood or other factors, we can think of Little Gotham as one big population. Now, we know that every Little Gothamite will not have a streetlight problem in a given year and we also know that everyone who has a streetlight problem will not bother to call 311. Let’s consider 311 calls to be a sample of problems and the city’s responses to those problems to be a sample of its responsiveness. Can we tell by examining the samples from Westchester and Scarboro whether these samples are likely to be from one population (i.e., a Little Gotham where every resident’s concern gets the same attention regardless of neighborhood) or from two different populations (i.e., one population featuring great responsiveness to residents and another population featuring less responsiveness)? Yes, we can tell by examining those samples. A calculation called a t-​test takes into consideration the mean, standard deviation, and standard error of the two sets of observations, and figures out the likelihood that the two sets were drawn from the same population (Table 9.2). The mean is a set’s average score. The standard deviation measures the extent to which the observations are tightly clustered or widely dispersed about the mean. The standard error of the mean is an

117

118

the basics

Table 9.2  Performing a t-​Test to Compare Two Groups When comparing two different groups with numerical data, there are three types of t-​tests one might use depending on the data. The first type is for data that is “paired” which means that the two groups are equal in size and are matched in some way. The second and third types of t-​tests can have groups that are unequal in size and hence are “unpaired.” We would use the second type of t-​test where we anticipate that the variance across the data is roughly equal; the third type assumes the variance may be unequal. The formulas are slightly different. Readers wanting details behind the formulas and calculations for the three types of t-​tests should consult a basic statistics textbook. In this table we will focus on the t-​test for unpaired data and an assumption of unequal variance, as it is the most conservative place to start. The t-​test formula assuming unequal variance is as follows:

t=

x1 − x2 s12 N1

+

s22

=

x1 − x2 se12 + se22

N2

To complete the necessary calculations we will need the means for the two sets of data, the standard deviations or variances, and a t-​statistics table.

Mean Standard deviation

∑ (x − x )

2

s=

Westchester

Scarboro

2.9 workdays

3.3 workdays

s=

n −1

1.90 10 − 1

= .459 Variance s2 = the square of the standard = .211 deviation Putting the data into the formula produces the following t-​score

t=

s=

3.60 10 − 1

= .632 = .400

2.9 − 3.3 = 1.618 .211 .400 + 10 10

The degrees of freedom is calculated using the following formula:

[ df =

s12 s22 2 + ] n1 n2

s12 2 s2 ) ( 2 )2 n1 n + 2 n1 − 1 n2 − 1

(

.211 .400 2 + ] [.0611]2 10 10 = = = 16.43 .211 2 .400 2 .02112 .04002 ( ) ( ) + 10 9 9 + 10 10 − 1 10 − 1 [

Given a t value of 1.618 and 16.43 degrees of freedom, the probability is greater than 10 percent that the observed difference is simply due to chance and therefore not a statistically significant difference.

me aningful diffe r e nc es in p e rformance

estimate, by calculation, of the amount of error that occurs when a sample’s mean is used to estimate the population’s mean. Using the t-​score and degrees of freedom2 an analyst can easily check a t distribution table (see Appendix B) and discover the probability that the difference between the two sets is simply a matter of chance. If the table reveals a substantial probability that observed differences in the samples are just a matter of chance, this suggests that nothing special is going on here—​ in the case being examined, that neighborhood really makes no difference. For instance, a probability of 50 percent (p = .50) would mean that there is a 50–​50 likelihood that the observed distribution happened by chance. One neighborhood might have been favored this time, but drawing observations a second time might produce a result favoring the other neighborhood. But finding a low probability that the results happened by chance would suggest that something special is going on—​in this case, that neighborhood matters. A low probability would indicate that one’s neighborhood influences responsiveness. For social science research, statisticians customarily consider a probability of 5 percent (p ≤ .05) to be the cut-​off for statistical significance. If the t-​test indicates that the probability is less than or equal to 5 percent that the two sets of observations came from the same population, then statisticians are willing to conclude that responsiveness to Westchester residents was significantly better than responsiveness to Scarboro residents. A word of caution to public sector managers and analysts is in order here. Bear in mind just how stringent the 5 percent cutoff point is, when thinking about applying it to a management or policy issue in local government. To use a cutoff of 5 percent would mean that you are willing to accept that differences in the sets of observations—​the differences in their means and dispersion—​ are significant only if the odds of such differences happening by chance are one in 20 or less. In the real world of local government, managers may find it prudent to respond even when the odds are less overwhelming. What if the probability that the observed distribution occurred simply by chance was not 1 in 20 (0.05) but 1 in 3 (0.33)? This would mean that there is a 2 in 3 chance that the factor under consideration may, in fact, be 2 Degrees of freedom is simply the sum of n − 1 for the first set of observations and n − 1 for the second set. In this example with 10 observations in each set, the degrees of freedom is 18.

119

120

the basics

associated with the occurrence. If there is a two in three chance that service discrimination might be happening, should not the manager respond by at least considering a deeper investigation? Must the odds really be greater than that?

BOX 9.1  WHEN ARE A STUDY’S FINDINGS SIGNIFICANT? When statistical tests are performed on a set of data, the results include not only the statistic itself but also the probability that this particular result might have occurred by chance. If there is a 50–​50 chance that the result could have happened by sheer luck rather than because of some special influence, then it is difficult to make the case that the influence in question was the cause. But if there is only a 5 percent probability that the result could have happened by chance, then there is a strong case that the result was influenced by the factor being examined. The piece of information among the results of a statistical test that will tell you the probability is the p value—​the estimated probability that the result could have occurred by chance. In traditional social science research the threshold of significance is p ≤ .05 or a probability less than or equal to 5 percent that the result could have happened by random chance. At this level, researchers are willing to accept that something other than a chance occurrence has happened. Sometimes a relaxed standard of p < .10 is deemed acceptable. The significance threshold is different for analyses performed for practical purposes in a local government setting. In academic studies researchers worry about declaring that something is associated with or perhaps even caused a result when it, in fact, is not or did not; hence they set the threshold low, so that all but the most extreme results will be deemed insignificant and ignored. In a local government environment, professional managers and elected officials will want to react if there is a 75 or 80 percent probability that something untoward is influencing an undesirable result. Consider a parks and recreation director who notices that more reported injuries are occurring at playgrounds with the equipment of a particular manufacturer than at playgrounds with other manufacturers’ equipment. If a statistical analysis of the differences showed that more injuries were associated with the equipment produced by the manufacturer of concern but the p value was .50—​meaning there was a 50–​50 chance that it occurred by chance—​the parks and recreation

me aningful diffe r e nc es in p e rformance

director might drop the matter. On the other hand, if p < .05 the director might check with the city attorney to talk about legal action against the manufacturer of dangerous equipment. However, suppose that p = .25—​ meaning that there is a 25 percent probability that the result occurred by chance. Although an academic researcher would drop the matter as statistically non-​significant, a professional manager should not. “If there is a 75 percent probability that this manufacturer’s equipment is more dangerous than others, we need to act,” she might declare. She might pursue action with the manufacturer—​perhaps requesting the installation of a safer model—​or she might have to consider this investment a loss and simply replace the equipment rather than risking further injuries. The significance threshold of management analyses in local government in most cases should be set at a more generous range than in academic studies. We will offer a suggested set of ranges and a pair of caveats. First, our significance ranges for practical administrative analyses: p < .25 Consider this to be a significant finding for practical administrative analyses. If there is a 75 percent probability that a result of concern is happening not by chance but because an underlying factor is influencing the result, some further investigation and possible corrective action may be warranted. .25 < p < .40 Consider a probability in this range to warrant discussion with advisers. Be cautious about asserting anything definitive here, because the evidence is not overwhelming. However, take into consideration the ramifications of mistakenly rejecting the hypothesized relationship of concern. Failing to act on findings even in this range could seem negligent in retrospect. p > .40 The results of this analysis do not warrant action. Next, our caveats: First, the above rules of thumb apply to special analytic studies undertaken infrequently, not to the daily, weekly, or monthly analyses of routine operating statistics. By their nature and frequency, the recurring analyses of operational metrics put them in a different category. An analysis repeated 52 times in a year or 260 times in five years is likely to yield a few results meeting the strictest academic thresholds of significance and many results meeting loosened thresholds. Many of these are likely to be false positives. For daily, weekly, or monthly analyses of operational metrics it is more appropriate to use either the traditional

121

122

th e basics

academic threshold (p < .05) or the testing standards of control charts (see Chapter 27 “Control charts”). Second, it may be important to consider the difference between statistical significance and substantive significance. For example, a study examining the quantity of work performed by plan reviewers equipped with a particular technology and the quantity performed by plan reviewers operating without that technology might detect a small but statistically significant difference between the two groups. If the average difference is only 0.2 plans per day, decision makers would still have to address the question of whether the difference is substantively significant enough to justify the new technology’s expense.

Interpreting the result in Little Gotham When Mattie shared the results with the city manager, she noted the more favorable average response times in Westchester for most types of service calls but pointed out that the differences were not all that great and none of the differences was statistically significant by conventional standards. “There’s a decent likelihood that the differences in response times to streetlight problems could have occurred by chance rather than by discriminatory practices. In fact, the probability of chance for these particular numbers (p = .125) is one in eight. That’s pretty far from the conventional standard for concluding that it’s not simply a matter of chance (i.e., a probability of .05 or less).” “I understand that these numbers don’t prove that service discrimination has occurred,” Cole conceded, “but the pattern across these service categories is troubling. I am going to follow up to be certain that the only consideration given to neighborhood location in the work scheduling process is the clustering of job sites for the sake of efficiency. We need to be sure that Westchester and a few other neighborhoods aren’t getting preferential treatment.” Several days later at the city council meeting, the city manager made a presentation reviewing the service response numbers for the Westchester and Scarboro neighborhoods—​numbers that had been included in the packets distributed to the council members a few days before the meeting.

me aningful diffe r e nc es in p e rformance

“As you can see, the average response times for Westchester are a little better for every category of service except one.” “Are any of these differences statistically significant?” asked Councilman Tom Frye, a researcher at a local laboratory. “No, not by conventional standards.” “Well, that’s good.” “But still, it’s a troubling pattern and we have begun some follow up just to be sure that everyone understands our commitment to equity. We are glad that any disparity appears to be small, but the pattern should be more random or there should be good explanations for why it’s not. We are going to monitor this closely and be certain that our practices are what they need to be.” Harold Jeffords sat in the audience four rows from the back of the council chambers with his arms folded across his chest and just a hint of a smile on his face.

Suggested for further information Eller, Warren S., Brian J. Gerber, and Scott E. Robinson. Public Administration Research Methods: Tools for Evaluation and Evidence-​Based Practice, 2nd ed. New York: Routledge, 2018. See “Descriptive Statistics,” Chapter 13, 270–​290. Meier, Kenneth J., Jeffrey L. Brudney, and John Bohte. Applied Statistics for Public and Nonprofit Administration, 9th ed. Stamford, CT: Cengage Learning, 2015. O’Sullivan, Elizabethann, Gary Rassel, Maureen Berner, and Jocelyn DeVance Taliaferro. Research Methods for Public Administrators, 6th ed. New York: Routledge, 2017.

Web resource For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

123

Part III ANALYZING A UNIT’S CAPACITY TO MEET THE DEMAND FOR SERVICE

A government’s ability to meet citizen demands for services is often constrained by its limited resources. It is difficult to meet all the needs and desires with available revenues. Sometimes, however, more prudent management of current resources could help narrow the gap. The first step in matching demands and capacity comes with understanding the patterns of demand for a given service and the patterns of resource deployment in addressing those demands (Chapter 10). If the demand and resource deployment patterns are misaligned, there might be realistic ways to bring them into better alignment. The strategy many departments and service recipients propose for meeting high demands is to just increase resources allocated to the service in question—​increased dollars, staffing, or both. This option completely ignores resource limitations, competing demands for the limited resources available, the possibility that current resources might be sufficient if only they were applied optimally, and the possibility that the real issue is not resources, but subpar performance. What are a few examples of suboptimal use of available resources? The distribution of a department’s work might not reflect its declared service priorities, might not deploy employee skills most wisely, or might even contribute to service backlogs (Chapter 11). The

126

analyzing capa city to m eet s er v ice d em a n d

possibility of subpar performance can be explored through comparison to performance standards (Chapter 12). The question of proper staffing to meet service demands is often more complex than proponents of simple population-​based staffing formulas might suggest. Careful consideration of a community’s service demands, its willingness to invest in labor-​saving equipment, its willingness to pay for higher-​level skills in its employees, and its preferences for in-​house or contracted services are all elements of staffing analysis (Chapter 13). If a decision is made to add an uninterruptable position to the staff (e.g., having an additional police officer on duty around-​the-​clock seven-​days-​ a-​week or an additional lifeguard on duty every minute the pool is open), a staffing factor calculation (Chapter 14) will provide the answer to the question: “How many employees do we need to hire to allow for days off and other predictable absences and still provide uninterrupted coverage of this position?”

10 DEMAND ANALYSIS

All organizations—​public or private—​are vulnerable to bureaucratic rigidity. They may become enmeshed in unnecessary or unnecessarily precise rules of operation, hamstrung by intentional or unintentional emphasis on employee rather than client convenience, or simply enslaved by force of habit: “We’ve always done it this way!” Well-​managed organizations attempt to get the best possible return for their investment of resources. Even long-​standing programs and practices in such organizations are subject to refinement or total revision. Sometimes such changes simply involve a better matching of resources and service demands—​ that is, marshaling necessary resources, including personnel, and having those resources available and functioning when they are needed most.

Scenario: Newbern, Ohio Employment practices of the city of Newbern, Ohio, conform to the standard pattern adopted by most other local governments with regard to hours of operation and workforce scheduling. Most employees work full-​time,

128

analyzing capa city to m eet s er v ice d em a n d

40-​hour schedules from Monday through Friday. Notable exceptions are found in the public safety operations, where police officers and firefighters maintain schedules that differ sharply from the norm. Except for administrative personnel, the employees of the Newbern Fire Department are assigned to one of three shifts. Each shift begins at 8:00 a.m. one morning, ends at 8:00 a.m. the next morning, and does not begin again until 48 hours later. This is commonly called a 24-​hour-​on/​48-​hour-​ off schedule. For example, the firefighters on Platoon A may report for duty at 8:00 a.m. Tuesday, complete their tour at 8:00 a.m. Wednesday, and report for duty again at 8:00 a.m. the following Friday. The shift beginning at 8:00 a.m. Wednesday is covered by Platoon B, and the shift beginning at 8:00 a.m. Thursday is covered by Platoon C. Firefighters in Newbern are assigned to one of four fire stations located in roughly equivalent quadrants of the city. Standard police shifts in Newbern are 8 hours in length. The morning shift runs from 7:00 a.m. to 3:00 p.m.; the evening shift, from 3:00 p.m. to 11:00 p.m.; and the midnight shift, from 11:00 p.m. to 7:00 a.m. All police officers operate from the single police station located in downtown Newbern. Most offices of the municipal government are located in Newbern’s city hall. A few, however, were relocated to an office building several blocks away because of crowded conditions in city hall. Almost all clerical and administrative employees report to work at 8:00 a.m., have an hour off for lunch at or near midday, and conclude their workday at 5:00 p.m.

A case of understaffing? For a couple of years, the city manager of Newbern has been convinced that the city government is understaffed. Several department heads have requested additional employees, citing numerous instances in which the current staff has been overwhelmed by a crushing workload. Several of those instances have been corroborated by telephone calls to the city manager from citizens angry about service delays or long lines at city hall. The police chief and fire chief are both pressing for additional personnel to bring their numbers a little nearer to those recommended for cities the size of Newbern by police officer and firefighter unions and advocacy groups.

d e m a n d a n a ly s i s

Last year the city manager rejected most requests for increased personnel. It was an election year, and he knew the city council would be especially resistant to recommendations that would require a tax increase. Department heads were encouraged to try to get by with current staffing levels. This year, however, when the city manager’s budget recommendations included moderate staff expansions—​increases that he felt were more than a year overdue—​he was startled to have them all rejected by the city council. Frustrated by the rejections and intent on getting the facts that might either suggest a solution to the problems of long lines and service delays or convince the city council of the need for additional employees, the city manager instructed an administrative assistant to collect data on the increasing workload in several city departments. After discussing workload problems with department heads and observing office and field operations, the administrative assistant suggested to the city manager that workload fluctuations might be a key aspect of the problem. While it was true that the departments were occasionally swamped by the workload, other periods seemed fairly light. Together, the city manager and administrative assistant decided to undertake a simple form of demand analysis. The results caused them to modify their thoughts regarding solutions to the problem.

Demand analysis In its simplest form, demand analysis is nothing more than a fairly detailed examination of workload and resource deployment patterns. A typical examination incorporates graphs that depict work volume and show how it varies over time or from one geographic location to another. Although line charts and bar charts (that is, histograms) are most common, simple frequency tabulations may also serve the analyst’s purpose. Incoming workload may be recorded by month in order to detect seasonal fluctuations, by day of the month, by day of the week, by time of day, or by geographic location. Similar charts or frequency tabulations are then compiled to depict resource deployment—​typically staff deployment—​for comparison with demand patterns.

129

130

analyzing capa city to m eet s er v ice d em a n d

Demand analysis has several important uses, including the following: •

•

•

Resource deployment. Once the pattern of demand is identified, the need for revised deployment may be clarified. Ideally, the deployment pattern for personnel and other resources will match the demand pattern. Geographic considerations. Careful consideration of facility location and strategies for service delivery can sometimes increase benefits to service recipients at little additional cost or even at a savings to the service provider. Adjustment of demand. In some instances, the service provider can influence changes in the demand pattern. The adjustment of fees for services (to discourage overconsumption) and the staggering of expiration dates for permits (to flatten the demand curve) are two examples.

Findings The demand analysis performed by the administrative assistant and city manager of Newbern caused them to challenge several of their initial assumptions regarding the need for more employees. For example, plotting calls for police service by hour of the day revealed a remarkable swing in the number of calls from one period to another (Figure 10.1). Calls were consistently heaviest from 2:00 p.m. to 1:00 a.m. and lightest from 3:00 a.m. to 7:00 a.m. This fluctuation revealed the inappropriateness of Newbern’s “constant manning” approach to deployment, whereby each shift had the same number of police officers assigned. The addition of just one police officer to the streets around the clock under the constant-​manning strategy, as previously proposed by the city manager, would require the employment of five new officers.1 The city manager decided, instead, to recommend establishing a fourth shift beginning at 2:00 p.m. and ending at 10:00 p.m. (Figure 10.2). This special shift would be staffed not by new officers but by 12 current officers—​four drawn from each of the three primary shifts. In this manner, 1 Each additional police officer would provide coverage for 40 hours per week, minus any time taken by that officer for vacation, sick leave, and other forms of absence. Because of such absences and because the police patrol function is a seven-​days-​a-​week activity, considerably more than three employees are needed to cover one position around the clock, seven days a week. See Chapter 14.

d e m a n d a n a ly s i s

Figure 10.1  Average Number of Calls for Police Service, by Hour of the Day Note: Based on statistics from the police department of Kettering, Ohio. Source: John S. Thomas, “Operations Management: Planning, Scheduling, and Control,” in Productivity Improvement Handbook for State and Local Government, ed. George J. Washnis (New York: John Wiley & Sons, 1980), 176. Copyright ©1980 by the National Academy of Public Administration. Reprinted by permission of John Wiley & Sons, Inc.

on-​duty strength would be reduced during off-​peak times and increased during peak periods at no additional personnel cost to the city.

Fire department A similar pattern was found regarding the incidence of fire alarms. Most fires occurred in the evening hours from 5:00 p.m. to 10:00 p.m. Initially, a strategy similar to that taken for police deployment seemed applicable to the fire department. However, because the fire company strength in Newbern is only three firefighters per engine, a reassignment that would reduce the response strength to two firefighters for some off-​peak alarms seemed inadvisable. Instead, the city manager would direct his strategy toward the possibility of hiring two or three new firefighters who would work 8-​hour shifts five days a week, covering the evening hours when fire incidence was historically highest. They would be assigned to stations with the heaviest demands. The number of firefighters proposed for the special shift was subsequently increased to four by a creative coalition of the fire chief and

131

132

analyzing capa city to m eet s er v ice d em a n d

Figure 10.2  Plan for Revised Deployment of Newbern’s Patrol Officers Note: This figure depicts the number of officers assigned to each shift, not the number on duty at a given time. Because of the need to provide two days off per week, plus vacation time, holidays, and occasional absences for illness and other reasons, the number of on-​duty officers at any given time will be lower. See Chapter 14.

recreation director. The peak period for recreation activity in Newbern occurs on weekday afternoons during the after-​ school program from 3:30 p.m. to 5:00 p.m. The recreation director finds it difficult to hire responsible supervisors for such short periods of time and has consistently been turned down in her requests for an appropriation for full-​time or more substantial part-​ time help. Consequently, the recreation director suggested that the two departments join forces: they would request authorizations for four new employees who would serve as recreation supervisors from 3:00 p.m. to 5:00 p.m. and as firefighters from 5:00 p.m. until 11:00 p.m. The fire chief agreed to give this shared arrangement a try.2 2 Fire departments tend to be strongly influenced by traditional practices; however, some have been creative in the assignment of nontraditional functions to qualified firefighters, and they often report favorable results.

d e m a n d a n a ly s i s

Customer service center Other mismatches in demands and resource allocations were discovered in office operations. Three years earlier, a customer service center had been established as a convenience for persons paying utility bills, purchasing plastic garbage bags, or pursuing other matters requiring permits or fees. Most of those functions had been concentrated in that single location. Four finance department clerks were assigned to cover the center’s counter, as needed, in addition to their other accounting duties. The demand analysis revealed that counter activity is light most mornings until about 11:30 a.m. and becomes light again after about 1:30 p.m. For two hours at midday, however, counter business is often heavy. The traditional scheduling of lunch breaks for the four employees makes an already difficult situation even more burdensome and contributes to complaints of long lines and slow service. With two of the four clerks taking their lunch hour beginning at 11:30 a.m. and the other two leaving for lunch at 12:30 p.m., only two employees are on duty during the peak counter demand period. Following discussions about the problem, the finance director proposed that the office move gradually—​through attrition in order to avoid layoffs or forced reassignments—​to a staff of two full-​time and four half-​time clerks. The authorized strength of 4.0 person-​years would not be affected, because two full-​time and four half-​time employees still equal 4.0 person-​years. Two of the part-​time clerks would work from 9:30 a.m. to 1:30 p.m., and the other two would work from 11:30 a.m. to 3:30 p.m. The four part-​time clerks would be assigned primary counter duty, thereby allowing the two full-​time clerks to work on accounting assignments without interruption unless absences forced them to cover counter duty temporarily. Because all four part-​time clerks would be on duty during the 11:30 a.m. to 1:30 p.m. time period, counter strength would be doubled during peak periods at no additional cost. In fact, the city of Newbern incurs few fringe benefit expenses for part-​time employees; thus, service responsiveness would be improved while total costs actually would be reduced.

Human resources department Demand analysis uncovered a similar problem in the human resources (HR) department. Although the department is small, consisting of the director,

133

134

analyzing capa city to m eet s er v ice d em a n d

one technician, and a secretary, it was among the units relocated to an office annex when city hall became overcrowded. In conducting demand analysis for the HR operation, the administrative assistant not only relied on departmental records but also spent several lunch hours over a four-​week period in the lobby outside the HR office and kept a log of job applicant traffic. He discovered that a substantial volume of job seekers arrives during the noon hour when all three staff members are away at lunch. This discovery not only prompted concern from a public relations standpoint but also raised the question of whether the city might be losing some of the best potential applicants. It seemed reasonable to assume that persons already employed elsewhere but wishing to improve their status would be likely to explore alternative opportunities during the noon hour. Unemployed persons could come anytime. When shown the demand statistics, the HR director was surprised by the volume of noontime traffic. He immediately expressed interest in improving the situation, although he preferred that his own time and that of the technician be left as free as possible for technical and administrative duties more demanding than receiving applications. Three options seemed most reasonable: (1) employment opportunities could be posted at the service center at city hall and applications received at that location; (2) a mini service center could be established at the office annex at least during the noon hour for services handled at that location, with staffing provided on a rotating basis by departments located there; or (3) the secretary and HR technician could take staggered lunches to assure service for job applicants during the noon hour.

Business license issuances Focusing attention on still another office operation—​demand analysis of business license issuances and renewals—​revealed an unbalanced workload (Figure 10.3). Most business licenses expire in November or July, creating a workload burden during those months. By staggering expiration dates throughout the year, the burden could be spread more evenly, allowing prompt action on renewal applications, causing less interference with other office duties, and eliminating the need for overtime. The results of demand analysis in the departments examined encouraged the city manager to request subsequent analyses for other departments and

d e m a n d a n a ly s i s

Figure 10.3  Demand Profile for Business Licenses and Permits Note: Based on information for business licenses and permits in the District of Columbia. Source: John S. Thomas, “Operations Management: Planning, Scheduling, and Control,” in Productivity Improvement Handbook for State and Local Government, ed. George J. Washnis (New York: John Wiley & Sons, 1980), 181. Copyright ©1980 by the National Academy of Public Administration. Reprinted by permission of John Wiley & Sons, Inc.

activities. Specifically, he wanted demand analyses completed for the civic center, the swimming pool, parks maintenance, and equipment repair activities. Demand variations at the civic center and swimming pool by season, day of the week, and time of day could suggest changes in the hours of operation of those facilities, as well as modified staffing levels and deployment patterns. Seasonal variation in the parks maintenance activities could suggest revised strategies for the use of seasonal help; the deferral of some activities, such as minor construction, to the off-​season in order to level out the workload; and perhaps merger or coordination with public works crews for the use of parks maintenance employees for snow removal and other off-​season activities. Analysis of the demand for equipment repair could reveal the introduction of high volumes of workload in late afternoons near the close

135

136

analyzing capa city to m eet s er v ice d em a n d

of the normal work shift for most departments. Vehicles brought in for repair or preventive maintenance might be unavailable for use the following day and perhaps longer, not because maintenance actually requires 24 hours or more but because the demand is initiated at a time when most mechanics are leaving work for the day. Such findings might justify establishing a second shift for equipment maintenance. Although a second shift could be staffed by adding mechanics, it could also be established without expanding employment by reassigning current mechanics or assigning new employees following attrition among current mechanics. Perhaps without increasing the total number of mechanics, the productivity of the equipment maintenance function could be improved by better matching the availability of mechanics with the time that vehicles are most available for repair or preventive maintenance.

Utility of demand analysis Simple demand analysis requires careful observation and tabulation but nothing beyond the capability of most administrative assistants. If adequate records are already on hand, it is an especially simple form of analysis with great potential usefulness. Many local government operations have been established under the apparent assumption that a balanced demand for services exists. That assumption frequently is incorrect. For instance, a service agency might be staffed from 8:00 a.m. to 5:00 p.m., Monday through Friday, in a pattern totally at odds with client preferences for evening and weekend service. Under such circumstances, demand analysis might reveal peak demand at or near the close of normal working hours—​the current operating hours closest to the client preference. Prime candidates for demand analysis would include operations whose clients frequently incur long waiting times for services; operations that may have grown somewhat lax or indifferent to their clients’ demands, whether those clients are internal to the organization (that is, other local government departments) or external; operations that stress symmetry and administrative convenience over functionality and productivity; operations that place a premium on employee rather than client preferences; and operations that

d e m a n d a n a ly s i s

simply have not recently compared patterns of demand and resource allocation and wish to assure themselves of a proper match.

BOX 10.1  OTHER APPLICATIONS OF DEMAND ANALYSIS Logical candidates for demand analysis include the following: • •

•

• •

•

a swimming pool that is open and staffed at 9:00 a.m. but draws few swimmers before 10:30 a.m. a library that consistently experiences difficulty closing at the announced time because large numbers of patrons are still in the building a parks maintenance department that has the same staffing year-​ round—​that is, an equal number of full-​time equivalent employees summer and winter a streets department that has the same staffing year-​round suppliers of water, gas, and electricity that encounter wide variations in demand and could be interested in designing strategies to level out peak demand times or seasons service calls—​ emergency and otherwise—​ in several government departments, as a step toward developing personnel deployment strategies that efficiently match call-​for-​service patterns

Advanced demand analysis How closely do staffing patterns and other resource allocation strategies match the demand for services? The graphic technique described in this chapter provides a “rough cut” demand analysis that in many cases is sufficient for detecting a demand-​to-​resource mismatch and devising a practical solution. But what about cases that require a finer analytic instrument to supplement the demand graphs? Local governments that want greater precision might consider calculating the correlation between the pattern of demand for a service and their pattern of resource allocation. Instructions for making such a calculation are provided in Chapter 6.

137

138

analyzing capa city to m eet s er v ice d em a n d

Reference Thomas, John S. “Operations Management: Planning, Scheduling, and Control.” In Productivity Improvement Handbook for State and Local Government, edited by George J. Washnis, 171–​203. New York: John Wiley & Sons, 1980.

Suggested for further information King, Norman R. “Manage Demand, Not Supply.” Governing 9, no. 12 (1996): 13.

11 WORK DISTRIBUTION ANALYSIS

Sometimes, problems in local government operations are symptoms of the improper alignment of program priorities and work assignments, a mismatch of tasks and employee skills, inadequate training, and poor employee deployment strategies.

Scenario: Weber, Pennsylvania The mayor of Weber, Pennsylvania, was enjoying his regular Saturday morning game of golf with three longtime friends and golfing buddies. By the third hole, the usual banter had begun to die down, and the mayor’s thoughts drifted back to city business. “Sam, you’re an officer with one of the biggest auto parts dealers in the region,” the mayor remarked. “I used to see your company’s name on our bid sheets all the time, but I don’t any more. Why have you stopped competing for the city’s business?”

140

analyzing capa city to m eet s er v ice d em a n d

“Simple. We weren’t winning any bids,” Sam replied. “Call it ‘sour grapes’ if you want, but it seemed to our people that your buyers drew up specifications that favored one or two of our competitors and never gave consideration to any factor other than purchase price. We think our parts are better than anything our competitors offer. They last longer and work better—​and we can document that claim. We have a better warranty. But it never seemed that those factors were given any consideration in the comparison of bids.” “Why haven’t you said anything about this earlier, Sam?” “I didn’t want you to think I was trying to get some special consideration because of our friendship. Besides, these are just the opinions of some of our sales people. I haven’t looked into it, personally.” “Well, you can bet that I will!” remarked the mayor. Back in the office on Monday morning, the mayor summoned an administrative aide. He reported his friend’s comments and asked if the aide had heard similar criticisms. “No, nothing like that. I’ve heard a few other complaints about our purchasing procedures—​you know, occasional gripes about equipment downtime and mechanics blaming it on inability to get parts, but that’s all I’ve heard.” The mayor asked the aide to investigate the situation and to let him know what he discovers. “While you’re at it, look into the problem you mentioned, too.”

Overly restrictive specifications? The aide knew that when the mayor asked for an investigation of a problem, he expected more than a report of the comments of persons connected with the operation in question. He wanted facts and figures. The aide began his investigation with a visit to the parts manager. He described the allegations regarding bid specifications drawn to favor particular vendors. The parts manager reacted angrily, but his comments failed to deflect the criticism. “Look, we just try to get the best parts we can. If we’ve had good luck with a part in the past, why shouldn’t the specifications reflect the features of that part? If other manufacturers want our business, let them make a product just like it.”

w o rk d i s t ri b ut i o n a n a ly s i s

The aide did not argue with the parts manager, but he knew that there was another side to that issue. Overly restrictive specifications might eliminate inferior products, but they could also eliminate good products that would perform the intended function as well as or better than the previously preferred product. A preemptive narrowing of the field could reduce the competitiveness of subsequent bids. The aide suspected that there might be substance to the comments made by the mayor’s golfing buddy. He decided to proceed in two directions. To determine the nature and magnitude of the problem, he collected a sample of two dozen bid specifications for major parts orders that he could review for restrictiveness.1 To determine how much attention was being given to the preparation of specifications and the comparison of vendor bids, and by whom, he decided to prepare a work-​distribution analysis. That analysis would also give him an opportunity to explore the criticism he had heard personally regarding equipment downtime for lack of needed parts. Review of the sample of bid specifications confirmed the allegations. Almost half unnecessarily restricted competition by specifying intricate design features rather than performance capabilities and characteristics. “Sure, this approach simplifies the job,” the aide muttered to a colleague. “They don’t have to develop a set of specifications. They just use their favorite manufacturer’s specs. And comparing bids is a snap! But our objectives should be product and price—​not how easy it is to come up with specs!”

Work-​d istribution analysis The aide was convinced that proper training or reorientation was needed for persons involved in the specification and bid comparison process. Furthermore, he wanted to assure himself that sufficient emphasis was being placed on that process—​both in allocating the proper amount of time

1 The aide decided that drawing a systematic sample would be satisfactory and most practical in this instance (see Chapter 5 “Sampling for analysis”). Upon his request, a list was prepared showing all parts orders during the previous 12 months that exceeded $5,000. The list had 245 entries, so he randomly selected a parts order from among the first 10 entries, marked that order and every 10th entry thereafter for a total of 24, and requested bid specifications from those orders for detailed examination.

141

142

analyzing capa city to m eet s er v ice d em a n d

and in assigning appropriate employees to that task—​to achieve output quality in keeping with its importance. With the assistance of the parts manager, the aide constructed a work-​ distribution chart for the vehicle maintenance parts room (Table 11.1). Such a chart can be developed for any operation or department without much difficulty, especially if guided initially by a person well acquainted with all the major processes or tasks involved in that operation or department. Although variation is possible, most work-​distribution charts follow a similar format: major processes or tasks are identified in the left-​hand column, all employees are identified by title across the top row, and the number of hours devoted by each employee to each task in a given period (for example, one week) is reported at the intersection of the appropriate row and column. The employees in the parts room were asked to record all their activities for one week, noting how much time was devoted to each of the task categories identified previously with the assistance of the parts manager. One week later, parts person 1 reported that he had spent 5 hours comparing vendor prices, 5 hours selecting vendors and placing orders, 23 hours distributing parts to mechanics, 4 hours updating computer records, 1 hour ordering new parts, and 2 hours spent on “undesignated and miscellaneous” duties. This included time spent on breaks or work other than the designated categories. Those figures, plus the numbers reported by the other nine employees, are recorded in Table 11.1. Each employee column is totaled and shows a full accounting for a 40-​hour workweek. In addition, rows are totaled to show “total staff hours” devoted to each task. Finally, the overall percentage allocation for each task is displayed in a column labeled “% Total.” Work-​ distribution charts frequently reveal shortcomings in work assignments that lie hidden until subjected to systematic analysis. Patricia Haynes notes five fairly common shortcomings: •

•

Work priorities may be misdirected or lacking altogether, as when too much time is spent on relatively unimportant tasks rather than on work associated with the principal objective. In government, overqualified personnel tend to be assigned to jobs that lesser skilled employees could perform, or employees are assigned to do work for which they lack the proper training.

newgenrtpdf

Table 11.1  Work-​Distribution Chart, Vehicle Maintenance Parts Room, City of Weber Process/​task description

% Total

36 40 22

9.0 10.0 5.5

61 88

15.3 22.0

12 7 6

3.0 1.7 1.5

23 12

5.8 3.0

12 23 9 15 34 400

3.0 5.8 2.3 3.8 8.5 100.2a

Parts Manager

Parts Person 1

Parts Person 2

Parts Person 3

Parts Person 4

Stores Clerk 1

Stores Clerk 2

Stores Clerk 3

8

5 9

5 5

3 2

4

23

5 27

29

4 1

4 1

20

29

7 9

5 3 2

4

3 4 2

2

7

Buyer 1

Buyer 2

10 21 5

18 11 6

4 40

5 40

3 1

12 23 5 10 3 40

2 40

3 40

2 40

5 40

3 40

5 40

4 5 2 40

a  Column does not sum to 100 percent due to rounding. Source: Adapted from Patrick Manion, “Work Measurement in Local Governments,” Management Information Service Report 6, no. 10 (Washington, DC: International City Management Association, 1974), 3; reprinted in Patricia Haynes, “Industrial Engineering Techniques,” in Productivity Improvement Handbook for State and Local Government, ed. George J. Washnis (New York: John Wiley & Sons, 1980), 207. Used by permission of the International City/​County Management Association.

w o rk d i s t ri b ut i o n a n a ly s i s

New vehicle parts purchases   Reviews orders, specifications   Compares vendor prices   Selects vendor, places order New vehicle parts distribution   Receives and stocks parts   Distributes parts to mechanics Tire control   Receives and stocks tires   Distributes tires to mechanics   Discards old tires Inventory maintenance   Updates computer records   Orders new parts Rebuilt parts control   Receives and stocks parts   Distributes parts Administration Supervision Undesignated and miscellaneous Total staff hours per week

Total staff hours

143

144

an alyzin g capa city to m eet s er v ice d em a n d

•

•

•

When employees are assigned to several unrelated tasks, problems can arise because more errors tend to occur when people are interrupted frequently or when they must switch to another, unrelated activity. Employees assigned too few tasks may become bored, lose interest, and perform poorly. Moreover, it may be difficult to find satisfactory substitutes when those employees are on vacation or sick leave. Assigning the same task to a number of employees can lead to problems, since individual styles differ and tend to result in inconsistencies in work procedures. This breeds excessive cross-​checking of work assignments and, unless guidelines and quality controls are enforced, leads to inconsistent results.2

FROM THE ELECTRONIC TOOLKIT BOX 11.1  CREATING WORK-​DISTRIBUTION CHARTS Administrators may prefer to use Microsoft Excel in the creation of work-​ distribution charts because the computer will do all the calculations, and, after the chart is prepared once, it can easily be used as a template and reused when other data are plugged in. To begin, enter the labels for each column and row. (For directions on opening Excel and entering data see Box 1.1.) Using Table 11.1 as an example, the label “Process/​task description” would be entered in cell A1. The tasks (“New vehicle parts purchases” and so on) would be entered from A2 through A20. To make the column wider so that the entire title can be read, move the cursor up to the line between column A and column B. Wait for the cursor to change into a black line with arrows on both sides of it. When that shape appears, click and hold it down, moving the mouse to drag the side of the column to

2 Patricia Haynes, “Industrial Engineering Techniques,” in Productivity Improvement Handbook for State and Local Government, ed. George J. Washnis (New York: John Wiley & Sons, 1980), 208. Copyright ©1980 by the National Academy of Public Administration. Reprinted by permission of John Wiley & Sons, Inc.

w o rk d i s t ri b ut i o n a n a ly s i s

a more appropriate width. Another way to alter the width of the column is to highlight the column, right-​click, and choose “Column Width” from the options. After entering the data labels, use other columns to enter the actual data or the actual observations. At the bottom of each column, all the observations may be added together. To do this calculation, use the SUM function. To find the sum of the first column of numerical data, enter “=SUM(B2:B20)” and the total will be calculated. (An alternative to typing this instruction would be to use Function choice fx from the formula toolbar.) If you wish to calculate totals for every column, you could enter this function manually for each of the columns or you could use a simpler method. With the shortcut method, simply highlight the cell that contains the function, then move the cursor to the lower, right-​hand corner of the cell and wait for it to change into a smaller black cross. Click and hold it down while dragging the mouse across the chart until reaching the last column of data. This action will copy the command to all the highlighted cells and also change all the addresses to the appropriate cell names. If a similar chart needs to be made for another set of data, simply change the data and the results of the calculations will change as well. To replicate the column of percentages in Table 11.1 (see the third column of that table), you would add the formula as “=B2/​$B$21” in cell C2 of the Excel spreadsheet, where B21 would be the sum total as described above. The use of “$” in the formula means the reference to B21 is absolute and will not change if this formula is copied down the column. When the new cell C2 is copied down to C3 it will copy as “=B3/​ $B$21”, the numerator is relative and moves down but the denominator stays fixed on B21 as an absolute reference. In this way the formula can be copied down the column rather than entered separately for each row. As described this formula will produce a number between 0 and 1. To change its formatting to a percentage simply select on the Home menu, the number formatting for percentage. This will include the “%” as part of the formatting. Alternatively, the formula can be rewritten for cell C2 as “=100*(B2/​$B$21)” which will produce an output without the percentage sign.

145

146

analyzing capa city to m eet s er v ice d em a n d

Findings The work-​distribution chart did not hand the administrative aide any solutions on a silver platter, but it did identify some areas for more detailed investigation. For example, is the reviewing of orders and specifications important enough to justify more than 9 percent of the total time? Why does the parts manager have no role in the comparison of vendor bids? And why does he devote 14 hours (35 percent of his time) to updating computer records and ordering new parts? Could those functions be handled by the stores clerks and buyers, respectively, allowing the parts manager to devote more attention to supervision, reviewing orders and specifications, and comparing vendor bids? The work-​distribution chart also sheds light on the intermittent difficulties encountered by mechanics in their attempts to obtain parts from the maintenance parts room. Most of the problems have occurred when one particular employee, parts person 4, is absent from work. Unfortunately, that seems to be happening more and more frequently, with increasing instances of the employee calling in sick. A review of the work-​distribution chart reveals that parts person 4 devotes almost all his time to receiving, stocking, and distributing rebuilt parts and that no other employee is engaged in those functions. Two questions seem worth exploring. First, why is not at least one other employee involved in this activity to provide backup capability in case of absences? Second, has the specialization of parts person 4 led first to boredom and then to recurring “sickness”? The aide also noted that six employees are involved in purchasing parts. Have all those employees been adequately trained to perform that function—​that is, are they all fully qualified? To ensure adequate skills for this function, should a third buyer be hired? Alternatively, should one of the parts person positions be converted to a buyer position?

Results The aide discussed his findings with the parts manager. Although the manager was defensive at first, he eventually softened his stance and agreed that increased efforts should be made to avoid unnecessarily restrictive specifications and that he should become more involved personally in the preparation of specifications and the comparison of vendor bids. He was

wo rk d i s tr i bu t i o n a na ly s i s

especially intrigued by the aide’s thoughts regarding work-​distribution strategies and agreed to explore revised work assignments as a possible remedy to some of the problems being encountered in the maintenance parts room.3 The aide was encouraged by the parts manager’s response and reported his findings and recommendations for less restrictive specifications and revised work distribution to the mayor. The mayor was pleased with the thoroughness of the aide’s work and was particularly impressed with the work-​distribution chart. “I want to talk to the parts manager myself before deciding on this,” the mayor said, “but I want you to know that I am inclined to accept your recommendations. This is very good work.”

Utility of work-​d istribution analysis Work-​ distribution analysis offers a useful method for examining the application of human resources in an organization and detecting possible problems. In local government applications, for example, it could reveal instances where police officers are devoting inordinate amounts of time to duties that could be handled by civilian clerks, library aides are attempting to perform librarian duties for which they are ill prepared, and parks maintenance employees are spending more time on equipment repairs than on mowing. Analysis of this type could prove beneficial for virtually every local government operation.

BOX 11.2  OTHER APPLICATIONS OF WORK-​DISTRIBUTION ANALYSIS In addition to its value as a means of checking the alignment of work priorities, tasks, and staff skills, work-​ distribution analysis is also a useful device for organizations attempting to implement activity-​based

3 In some cases, civil service regulations or union rules may restrict managerial discretion regarding work reassignments. Even in such cases where negotiation is required, systematic analysis of operations can still play an important role. Careful analysis that identifies crucial factors restricting performance will help management establish its negotiation priorities and will help document the basis for its position.

147

148

analyzing cap a c ity to meet s er v ice d em a nd

costing or embarking on other efforts to identify the full costs of various operations (see Chapter 19). Such efforts depend on a reasonable level of precision in identifying the amount of time and other resources devoted to each activity or service. Work-​distribution analysis can be helpful in that regard.

References Haynes, Patricia. “Industrial Engineering Techniques.” In Productivity Improvement Handbook for State and Local Government, edited by George J. Washnis, 204–​236. New York: John Wiley & Sons, 1980. Manion, Patrick. “Work Measurement in Local Governments.” Management Information Service Report 6, no. 10. Washington, DC: International City Management Association, 1974.

Suggested for further information Brimson, James A., and John Antos. Activity-​Based Management for Service Industries, Government Entities, and Nonprofit Organizations. New York: John Wiley & Sons, 1998. Morley, Elaine. A Practitioner’s Guide to Public Sector Productivity Improvement. New York: Van Nostrand Reinhold Co., 1986. See “Improving Productivity through Work Redesign,” 89–​117.

Web resource For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

12 USING PERFORMANCE STANDARDS

A local government official attempting to rate the performance of a particular department or operating unit may do so from any of several points of reference. A subjective assessment may be drawn from impressions, hearsay, the comments of the department’s supervisor or the supervisors of interacting departments, the nature and volume of citizen compliments and complaints, or the official’s own personal “feel” from occasional direct observation or from the individual experiences of family members and friends. Such assessments may be strongly held; they may also be incorrect. A bit more objective are assessments based on expenditures and workload volume. Managers may contend, for example, that they have remained “within budget” for the last six or seven years in a row and are spreading more asphalt or processing more paperwork than ever before. Such assessments, however, may say little about the efficiency (unit costs or units per labor hour) or effectiveness (quality or usefulness) of the service. Officials best able to assess performance are those who can compare the efficiency and effectiveness of an operation with some meaningful benchmark. A jurisdiction with a good system of performance measures can compare

150

analyzing capa city to m eet s er v ice d em a n d

its performance against that of other jurisdictions or simply against its own performance at an earlier point in time. As an alternative or a supplement to such comparisons, it may compare actual performance with some target or established standard. A target may be set to meet or exceed historical performance or to beat the national or state average. A “standard” may be established using industrial engineering techniques or, in some cases, may be drawn from published sources. For example, standards are published for such functions as road maintenance, grounds maintenance, refuse collection, janitorial services, and vehicle repair.

Scenario: Armadillo, Texas The city manager of Armadillo, Texas, has had her fill of bickering between the city’s automotive garage and the various departments that operate vehicles repaired and maintained there. She long ago became weary of the charges and countercharges of the principal combatants: the vehicle-​ operating departments claiming that the mechanics were slow, inept, and poorly managed, and the garage staff countering that the departments were unreasonable and that operators abused the equipment. She had tried to address the problem from time to time, but for the most part only patches had been applied to this festering situation. Today the battle of the city garage spilled past its normal limits. This morning the city manager found herself refereeing an argument between the police chief and the budget director over the chief’s plan to have the department’s squad cars repaired and maintained at a private garage. Like many other local governments, the city of Armadillo finances its garage through fees assessed to the user departments. The police chief considered the fees excessive and wanted to take his department’s business elsewhere. “The city garage’s rates are too high, and its work is not that good,” he snapped. “My plan will save the city money. I can get better service at a lower cost by having the cars repaired elsewhere.” “That won’t save a dime! In fact, it will increase the city’s expenditures,” responded the budget director. “We will still have to pay all the mechanics’ wages, and, without the police department’s business, we’ll have to raise everyone else’s rates.”

u s in g p e rf o rm a n c e s ta n d a rd s

“Without the police department’s business, you can cut costs by getting rid of some of those lazy mechanics,” retorted the police chief. It was clear to the city manager that this problem would not go away and would not be resolved by patchwork treatment. She asked the police chief to be patient for six months while she searched for a suitable long-​ term solution. The city manager told the budget director that she wanted a thorough investigation of the problem and a recommended remedy. “If we decide to continue operating a city garage,” she said, “I want to have the facts that will show me it is operating efficiently.”

Work standards for mechanics Some local government jobs have many routine elements that recur frequently enough to make the development of performance standards possible. Essentially, a performance standard answers the question: How long should it take a competent employee to perform this task? The budget director found that for mechanics in private garages this question was being answered by performance rates published in commercially produced manuals or on compact disks. These manuals and CDs say, for example, that a competent mechanic should be able to remove and replace the spark plugs in a 2020 Ford F-​150 pickup and perform the tasks associated with this job in about 1.6 hours. Upon further investigation, the budget director discovered that several local governments scattered across the country were applying in one form or another mechanics’ standards based on the commercial manuals. He also learned that the adoption of such standards would probably not be a popular move with the city garage.1 The fundamental concept underlying the use of performance standards is that a city mechanic should be expected to perform at the same proficiency as a private garage mechanic. A job requiring 1.6 hours of a private mechanic’s time should require approximately the same from a city mechanic.

1 An account of union opposition to the use of work standards for mechanics in New York City is found in Frederick O. Hayes, Productivity in Local Government (Lexington, MA: Lexington Books, 1977), 231.

151

152

analyzing capa city to m eet s er v ice d em a n d

AVOID THIS COMMON ERROR BOX 12.1  BEWARE OF SELF-​SERVING INPUT STANDARDS Some so-​ called standards directly address resource inputs—​ but address performance only indirectly, if at all. Prescriptions declaring the amount or percentage of resources that should be directed to the library, to parks and recreation, or to other particular functions of local government fall into this category. So do declarations prescribing the number of police officers or firefighters needed per 1,000 population. Although often well-​intentioned, these prescriptions typically are promulgated by professional associations focused on a favorite local government function and serve the interests of association members and advocates, while failing to establish true standards of performance.

Implementation plan The use of performance standards would allow the performance of city mechanics to be judged on the same basis as private garage mechanics. If city mechanics could meet or beat those standards, current criticisms would be considered groundless. Accordingly, the budget director proposed the following steps: 1. The superintendent of the city garage would secure copies of at least two of the three most commonly used rate manuals or CDs for mechanics.2 City mechanics would be allowed to use as their standard the more lenient of the two publications for any particular repair work.

2 Among the most popular of the rate manuals are the three known informally as the Chilton Manual, Mitchell Manual, and Alldata Manual, which are revised annually. See detailed information about Chilton Labor Guide, Mechanical Labor Estimating Guide, and MOTOR Labor Guide Manual in the section, “Suggested for further information,” at the end of this chapter.

u s in g p e rf o rm a n c e s ta n d a rd s

Table 12.1  Excerpt from City of Armadillo’s Customized Standard Time Manual Ford F-​150 Pickup Maintenance task Basic inspection and road test Ford F150 Pickup, 2016–​2020 Basic inspection includes checking the horn, ignition switch, lights, starter operation, transmission engagement and operation, speedometer, and gauges for temperature, fuel pressure, oil, etc. Inspector will also examine pedal pads, door catches and cushions, glass, mirrors, wipers, and tire condition. Replace plugs Ford F150 Pickup, 2016–​2020 Remove and replace spark plugs. Check gaps. Inspect wires/​boots. Replace wires, if needed. Chassis and brake inspection Ford F150 Pickup, 2016–​2020 Check king pins and bushings, drag link and toe in. Inspect master cylinder. Check power take-​off shaft and bearings, hydraulic pump, etc., for wear and leaks. Inspect exhaust system. Check springs and shocks. Inspect drive line and U joints, operation of clutch and pedal clearance. Check brake operations. Remove wheels. Inspect brake linings.

Standard labor hours 1.1

1.6

1.9

2. The garage superintendent would prepare a condensed rate manual drawn from the commercial manuals but tailored to the particular types of equipment in the city fleet. The condensed manual would simplify administration of the new performance standard system (see Table 12.1). 3. The repair order form for garage work would be modified to include a column for standard times as well as actual time consumed by the work (see Figure 12.1). 4. The department’s computerized management information system would be modified to allow entry of standard time and actual time to allow the garage to produce performance reports by mechanic (see Table 12.2), by shift, by the garage as a whole, by vehicle category, and by repair type (see Table 12.3). 5. Additional repair time required on vehicles that were improperly or incompletely repaired the first time would not be counted as a second repair job but instead would be added to the first job. A standard

153

newgenrtpdf

154

Work Order No.

Repairs to be Performed

Standard Hours

Actual Hours

Date

Mechanic

YES

NO

Equipment License Tag Equipment Number Make Model Odometer Driver Complaint Name Department Phone Job Completion Notes:

Total Downtime Days: Approval by Supervisor:

Figure 12.1  Armadillo City Garage Equipment Work Order

Mechanic's notes:

Road Test

analyzing capa city to m eet s er v ice d em a n d

Date

newgenrtpdf

Table 12.2  Excerpt from Performance Report, City Garage, September 13–​17: Efficiency Ratings, by Mechanic Date

Work order

Repair description

Standard hours

Actual hours

Efficiency rating (%)

Bosquet, Buster

091721 Total 091321

34002

Exhaust

33807 33812 33814 33816 33820 33822 33815 33827 33830 33832 33833 33834 33835 33840 33842 33845 33849 33853 33856 33857 33858

Check engine light Brakes Emissions Manifold Alternator Emissions Wheel align Rear brakes Brakes Check engine light Oil change Check engine light Radiator/​thermostat Emissions Universal joint Struts Ignition Brakes Shocks Hydraulics Wheel align

0.8 29.4 1.0 2.2 0.7 0.9 0.9 0.7 2.4 1.6 2.2 1.0 0.5 1.0 1.4 0.7 1.2 1.3 0.9 2.2 0.4 2.2 2.4 27.8

1.1 30.2 1.1 2.3 0.7 0.9 0.6 0.7 3.2 1.4 2.0 0.8 0.5 0.8 1.2 0.7 1.0 1.3 0.8 2.0 0.3 2.8 3.1 28.2

72.7 97.4 90.9 95.7 100.0 100.0 150.0 100.0 75.0 114.3 110.0 125.0 100.0 125.0 116.7 100.0 120.0 100.0 112.5 110.0 133.3 78.6 77.4 98.6

Eberhart, Babe

091421

091521

091621

091721

Total

u s in g p e rf o rm a n c e s ta n d a rd s

Mechanic

155

156

analyzing capa city to m eet s er v ice d em a n d

Table 12.3  Excerpt from Performance Report, City Garage, September 2021: Efficiency Ratings, by Repair Type Mechanic

Date

Work order

Repair description

Standard hours

Actual hours

Efficiency rating (%)

Bosquet, Buster

090921

33805

1.0

1.1

90.9

Bosquet, Buster

092321

33860

1.0

1.2

83.3

Bosquet, Buster

092421

33861

1.0

0.9

111.1

Bosquet, Buster

092821

33911

1.0

1.0

100.0

Corry, Cecil

091321

33817

1.0

0.9

111.1

Corry, Cecil

092021

33872

1.0

0.8

125.0

Corry, Cecil

092821

33912

1.0

0.9

111.1

Eberhart, Babe

091321

33807

1.0

1.1

90.9

Eberhart, Babe Eberhart, Babe

091521 091521

33832 33834

1.0 1.0

0.8 0.8

125.0 125.0

Eberhart, Babe

092221

33902

Check engine light Check engine light Check engine light Check engine light Check engine light Check engine light Check engine light Check engine light Check engine light Check engine light Check engine light

1.0

1.0

100.0

2-​hour repair job that consumed 1.5 hours initially and 2 more hours subsequently to do the job properly would be recorded as 3.5 hours—​ or 1.5 hours in excess of standard time. 6. Outstanding performance relative to standard rates would be recognized and rewarded.3

3 Most cities have the option of rewarding a superior mechanic through performance evaluation and merit pay increase. Depending on state law and local ordinances, some may also have the option of special incentives, such as bonus payments or the grant of extra leave when requirements for a fair day’s work have been met (for example, in some cities operating with a “task system,” workers performing more than 80 “standard hours” of work in a two-​week period are awarded extra vacation time for use during a subsequent period, even if the actual time worked is less than 80 hours).

u s in g p er f o rm a n c e s ta n d a rd s

The city manager was delighted with the budget director’s recommendation. Not only did the plan promote greater efficiency, it also provided a means of monitoring garage performance against a widely accepted standard.

Results Armadillo officials quickly discovered that the success of their new system depended upon competent and conscientious administration. Work orders had to be prepared carefully and records maintained accurately. The most apparent initial cost of the new system was an increased administrative workload. A second “cost” came in the form of employee dissension. Several mechanics, including some who had been with the city for many years, expressed displeasure with the new system. They complained that city management did not trust them to perform a fair day’s work and insisted that city equipment was too specialized for private garage standards to apply. The dissension reached its peak shortly before implementation but began to subside as most mechanics discovered that they could attain the standards. Some even seemed to take pride in their performance compared with the standards and compared with the performance of their coworkers. Lingering opposition was confined primarily to those mechanics who had complained most vehemently at the outset and those whose performance inadequacies were revealed for the first time by the new system. The benefits of the new performance measurement system far outweighed its cost. Gradually the city garage began to meet most of the standards and even to surpass some of them. The police chief still complained about garage performance, but the city manager could now refute many of the chief’s claims of excessive repair time. She could also pinpoint problems when the claims were valid.

BOX 12.2  OTHER APPLICATIONS OF PERFORMANCE STANDARDS Engineered standards for vehicle maintenance provide a useful means of monitoring and evaluating the performance of mechanics. Other standards are relevant when the focus is on different functions of local government or other types of local government employees. For example, the government of Fairfax County, Virginia, and the city of Austin, Texas,

157

158

analyzing capa city to m eet s er v ice d em a n d

assessed their fire departments using national standards for fire service responses. The National Fire Protection Association (NFPA) has established ambitious standards for the performance of fire departments and related operations in response to emergencies. Each standard has its own identifying number (for example, NFPA 1410, NFPA 1561, and NFPA 1710, among others). NFPA 1710 calls for turnout times of 80 seconds or less (the time from dispatch until departure from the station) and 240 seconds travel time or less for the first engine company’s arrival at the scene in at least 90 percent of all incidents.1 Adding these times together, the first-​arriving company would arrive within 320 seconds (5 minutes and 20 seconds) of when it was dispatched. Full suppression response should be on the scene within 9 minutes and 20 seconds (80 seconds for turnout and 480 seconds for travel time), according to NFPA 1710. Local fire departments can judge their own performance by comparing the percentage of incidents in which they have met these standards to the NFPA recommendation of meeting them at least 90 percent of the time. Fairfax County reported its performance compared to the NFPA standards for the first-​arriving unit (5 minutes, 20 seconds) and full suppression response (9 minutes, 20 seconds) (Table 12.4). The Table 12.4  Fairfax County Fire and Rescue Department Performance Compared to NFPA Standards Indicator

Prior year actuals

Current estimate

Future estimate

FY 2016 FY 2017 FY 2018 FY 2019 FY 2020 Actual Actual Estimate/​Actual Service quality Fire suppression response rate for arrival of an engine company within 5 minutes, 20 seconds (National Standard: 90%) Fire suppression response rate for 15 personnel within 9 minutes, 20 seconds (National Standard: 90%)

50.69% 50.88% 52.00%/​ 48.82%

52.00% 52.00%

81.40% 82.18% 85.00%/​ 79.02%

85.00% 85.00%

Source: Fairfax County, Virginia, FY 2020 Adopted Performance Measures. Accessed on May 29, 2020, www.fairfaxcounty.gov/​budget/​fy-​2020-​adopted-​performance-​measures-​pm

u s in g p e rf o r m a nc e s ta n d a rd s

Figure 12.2  Austin Fire Department (AFD) Performance Compared to NFPA Standards Source: City of Austin, Texas. FY2020 Approved Budget, 106. department was approaching compliance with the 90 percent standard for full suppression responses, but was still well short of the standard for the first-​arriving unit. NFPA 1710 also specifies time standards for persons receiving and processing the initial 911 call—​call answering time (40 seconds) and alarm processing time (106 seconds). Adding these time increments to turnout time (80 seconds) and travel time (240 seconds) brings the total response time standard from the initial 911 call until arrival of the first company to 466 seconds—​just a little less than 8 minutes. Austin’s fire department monitors its performance by tracking the percentage of emergency responses within 8 minutes (Figure 12.2). 1 National Fire Protection Association, NFPA 1710: Standard for the Organization and Deployment of Fire Suppression Operations, Emergency Medical Operations, and Special Operations to the Public by Career Fire Departments (Quincy, MA: NFPA, 2020), 9, 10, and 26.

Utility of performance standards The availability of industry standards for vehicle maintenance and repair was a fortunate break for the officials of the fictional community of Armadillo, Texas. The development of performance standards from scratch can be tricky. Errors may have adverse consequences. Standards set too high may frustrate employees; standards set too low may be difficult to change. Poorly devised

159

160

an alyzin g capa city to m eet s er v ice d em a n d

standards or inadequately monitored systems may even yield perverse responses. For example, sanitation workers judged only on the weight of garbage collected might be tempted to water down their loads before having them weighed, or police officers judged solely on number of arrests might become overzealous and thereby generate complaints of police harassment. When standards derived from industrial engineering techniques already exist and have been published, many of the potential pitfalls can be minimized. Whether used for monitoring performance on a job-​by-​ job basis or more generally for projecting, for budgetary purposes, the amount of work a group of employees should be able to perform in a year’s time, published standards for common local government functions, such as automotive repair and maintenance, custodial services, street and grounds maintenance, and refuse collection, have considerable value.

BOX 12.3  USE OF DIFFERENT KINDS OF STANDARDS A panicked caller trying desperately to reach 911 wants their call answered immediately. An unanswered call ringing again and again only adds to their desperation. The state government of California expects all local governments in that state to answer at least 90 percent of these calls within 10 seconds. Oakland’s police department assesses the emergency communications unit’s 911 call answering performance by comparing it to the state standard (Figure 12.3).

Figure 12.3  Oakland’s 911 Call Answering Performance Compared to the State Standard Note: 2014 data Source: City of Oakland/​Office of the City Auditor. 2017. OPD Communications Division: 9-​1-​1 Call Operations Audit. Oakland, California: Office of the City Auditor, November 2, p. 5.

u s ing p e r f o rm a n c e s ta n d a rd s

Figure 12.4  Austin’s Drinking Water Turbidity Compared to the State Standard Notes: (1) NTUs are Nephelometric Turbidity Units. (2) AW is Austin Water. Source: City of Austin, Texas. FY2020 Approved Budget, 109.

Some standards that are of use to local government officials focus entirely on the final product of government activities, not the activities or work processes of individuals or teams involved in producing the product. Standards of water quality provide a good example. Citizens want a dependable supply of tasteless and odorless water that contains nothing that is harmful to their health. Federal and state environmental agencies go much further than that and regulate the maximum allowable levels of a variety of contaminants. Local water treatment departments can monitor their own performance by comparing the amounts of these contaminants in their water to the standards. Some governments report statistics on contaminants and other water characteristics in reports to citizens. One of the characteristics often reported is turbidity (measured in Nephelometric Turbidity Units (NTU)—​ a measure of particulate matter in a sample of water which, in effect, refers to the cloudiness of water. Too much turbidity can protect bacteria from the disinfectant effects of chlorine. An NTU of less than 1.0 is generally not detected by the naked eye. The US Environmental Protection Agency requires that at least 95 percent of all water samples in any month go no higher than 0.3 NTUs and that no sample ever goes higher than 1.0 NTU.1 Many state environmental agencies have the same standard. The water system for the city of Austin, Texas—​Austin Water (AW)—​ tracks its turbidity levels against the state and national standards of 0.3 NTUs and shows turbidity much lower (more favorable) than the standard (Figure 12.4). 1 US EPA, National Primary Drinking Water Regulations. United States Environmental Protection Agency. Accessed June 19, 2020, www.epa.gov/​ ground-​water-​and-​drinking-​water/​national-​primary-​drinking-​water-​regulations.

161

162

analyzing capa city to m eet s er v ice d em a n d

References Hayes, Frederick O. Productivity in Local Government. Lexington, MA: Lexington Books, 1977. National Fire Protection Association. NFPA 1710: Standard for the Organization and Deployment of Fire Suppression Operations, Emergency Medical Operations, and Special Operations to the Public by Career Fire Departments. Quincy, MA: NFPA, 2020. US Environmental Protection Agency. National Primary Drinking Water Regulations. Washington, DC: EPA. Accessed June 19, 2020. www.epa. gov/​ground-​water-​and-​drinking-​water/​national-​primary-​drinking-​water-​ regulations.

Suggested for further information Alldata Repair. Elk Grove, CA: ALLDATA LLC, updated annually. www.alldata. com. Ammons, David N. Municipal Benchmarks: Assessing Local Performance and Establishing Community Standards, 3rd ed. Armonk, NY: Routledge/​M.E. Sharpe, 2012. Chilton Labor Guide. Clifton Park, NY: Thomson/​Delmar Learning, updated annually. www.delmarlearning.com. Department of Defense. Roads, Grounds, Pest Control & Refuse Collection Handbook: Engineered Performance Standards. Document no. NAVFAC 0525-​LP-​156-​0016. Washington, DC: US Government Printing Office, 1984. Department of the Navy. Janitorial Handbook: Engineered Performance Standards. Document no. NAVFAC 0525-​LP-​142-​0061. Washington, DC: US Government Printing Office, 1987. Means Site Work Cost Data. Kingston, MA: R. S. Means Co., updated annually. Mechanical Labor Estimating Guide. San Diego, CA: Mitchell International, updated annually. MOTOR Labor Guide Manual. Troy, MI: Hearst Business Communications, Inc., updated annually.

13 STAFFING ANALYSIS

How many police officers, firefighters, or public works employees does a local government need? How many employees is the right number for a particular municipal or county government function? Rarely is there a simple, pat answer to these questions. An analyst attempting to find the appropriate staffing level for a given function must consider not only the service demand for that function but also the influence of policies and management strategies that affect staffing needs. Service demand refers to the volume of work a department or work unit is called to perform. Does the accounts payable division of a given local government handle a high volume of transactions or a low volume? Does the community have a high percentage of young families with school-​age children who place heavy demands on recreation programs and facilities? Or is it a population with a different age profile and fewer demands for recreational opportunities? The staffing needs of the accounts payable division and the recreation department will be influenced by the level of service demand.

164

analyzing capa city to m eet s er v ice d em a n d

Community policies and management strategy can also influence a local government’s staffing needs. Consider, for instance, the staffing ramifications of these policies and strategies: •

•

•

•

Service quality. A community’s decision to provide a given service at a particular quality level—​the highest quality, average quality, or minimum quality—​affects staffing requirements. Consider, for instance, the staffing ramifications of the decision to provide curbside collection of recyclable materials instead of offering a receptacle at a single drop-​off location for recyclables. Scope of service choices. The array of services provided by police, public works, or any other department in one community often differs from the array deemed necessary, desirable, appropriate, or affordable by community leaders, government officials, or even residents in another community. Refuse collection services in one community, for instance, might include special collections of brush and yard trimmings, recyclables, and even household appliances, while another community might limit its collections to household waste. In some cases, the scope-​of-​service differences are attributable to service delivery agreements between city and county governments, expanding the scope of services handled by one of the agreement partners and reducing it for the other. These scope-​ of-​service differences affect a department’s staffing needs. Furthermore, local governments that strategically engage in service load shedding are taking more dramatic steps to alter their staffing needs. Investment in capital equipment to reduce labor intensity. Many local government functions are labor-​ intensive. Performing these functions in a traditional manner requires the efforts of many employees working many hours. In some cases, however, a local government might choose to depart from traditional ways and divert part of its expenditure from employee wages to the purchase of labor-​saving equipment. For example, a solid waste department that has switched from older-​style trucks and three-​ person crews to one-​ operator trucks having a hydraulic arm to grab refuse containers and empty their contents will need fewer refuse collection employees. Fewer but better-​paid staff members. Some local governments choose to build their staffs with fewer but better-​paid employees. The operating premise of this strategy is that better compensation will enable the government to attract and retain higher-​caliber employees and that fewer but more capable employees will be able to perform the work of a larger

s taff i n g a n a ly s i s

•

number of lesser-​skilled employees. Governments making this choice expect an analysis of staffing levels to place their departments near the lower end of the adequate staffing range. Contracting for services. Local governments that contract for a portion of their services need fewer employees than those that handle everything with their own employees.

The analysis of staffing needs must take service demand, service policies, and management strategies into consideration.

BOX 13.1  ACHIEVING A FULL WORKLOAD: WHAT’S OPTIMUM? Some supervisors say that they strive to operate at 100 percent of capacity, squeezing every penny from each labor dollar. More than a few managers even proudly proclaim that their employees give 110 percent! Setting aside the mathematical impossibility of the latter assertion, industrial engineers provide more helpful insights regarding productivity potentials and practical implications of employee workload. Normal fatigue affects all workers, preventing them from maintaining top performance without pause. Even when industrial engineers determine a good work pace, they know they cannot expect this pace throughout the workday. Accordingly, they include a personal time, fatigue, and delay (PF&D) allowance in all their calculations. A typical PF&D allowance ranges from 9 to 15 percent, which suggests that supervisors should more appropriately strive for 85 to 91 percent of workers’ time used productively—​even if it doesn’t have the nice ring of a 100 percent target. For some categories of local government occupations—​for instance, police officers and frontline emergency medical service workers—​still other ratios may be relevant. In recent years, many communities have claimed community-​oriented policing as their operating mode. Although some perhaps have adopted little more than the label, others have committed to a public safety philosophy that requires extensive interaction with community residents and business owners—​a level of interaction that is not possible if all of the officers’ time is directed toward responding to calls for service, court appearances, report writing, and administrative chores. Many of these communities track something

165

166

analyzing cap a c ity to meet s er v ice d em a nd

called patrol availability factor, which is the percentage of time officers have available for undirected patrol (that is, the patrol availability factor is found by subtracting the percentage of the workday devoted to responding to calls, performing administrative chores, and carrying out various other duties). Many community-​oriented policing enthusiasts strive to achieve a patrol availability factor of approximately 40 percent. An important measure of operating efficiency in emergency medical services is unit hour utilization (UHU). An individual UHU is the ratio of time that an EMS unit is engaged on calls compared with the total on-​ duty time of the unit. The departmental UHU is the figure for all units combined. If a unit was on-​duty for 24 hours and was engaged on calls for 8 of those hours, its UHU was 0.33 for that period. Although EMS experts differ somewhat on their UHU prescriptions, some suggest a desirable range of 0.25 to 0.50 with several pointing to 0.40 as the optimum.1 Rates higher than 0.50 risk overuse and employee burnout, lack of available units during simultaneous emergencies, and inadequate preparation for the next call. Rates lower than 0.25 signal underutilization and inefficiency. UHU calculations are precise in some reports and approximate in others, with approximations often based on an assumption of 1 hour per call. Using this approximation, a UHU of 0.4 is achieved by a unit handling two calls every 5 hours. Pursuit of a favorable UHU must be balanced with other EMS concerns, including adequate response time and patient well-​ being. Expanding an EMS unit’s response zone to increase its UHU might be ill-​advised if in doing so average response time and patient save rates are impacted adversely. 1 See, for example, Olympia (Washington) Fire Department: 2004–​2014 Fire and Emergency Medical Services Master Plan (Boulder, CO: Public Safety Consultants, 2005), 144; Performance Audit: EMS (Pittsburgh, PA: City of Pittsburgh, Office of City Controller, 2008); “Land Ambulance Services,” (Ontario, Canada: Ontario Municipal CAO’s Benchmarking Initiative, n.d.), 10.

Scenario: Crab Harbor, Maryland Crab Harbor is a relatively affluent community that has enjoyed population growth and an expanding tax base in recent years. The city’s chief

s taffi n g a n a ly s i s

of police, William Ralston, is a local fixture. With the election of Tommy Carlucci as mayor last fall, the chief is now serving in his third mayoral administration. Budget requests are due next week, and Chief Ralston is taking the opportunity to give the new mayor a preview of the costliest item on his wish list. “We’re understaffed, Mayor,” Chief Ralston began, “and that’s damaging our ability to keep this city safe. It’s also putting officers at risk. We really need at least 20 more officers, but I am asking for only 14 more in next year’s budget.” “You’re kidding me, aren’t you, Bill?” Mayor Carlucci replied. “We don’t have the money for that kind of increase and I am not planning to push through a big tax increase with my very first budget.” “We’re just barely keeping the lid on crime in this community,” the chief insisted. “The national standard is two officers per 1,000 population, and we are way below that. Adding 14 would at least get us close.” “I will get back to you on this, Bill. But don’t count on getting 14 additional officers next year.”

Analyzing staffing needs Department heads and citizen advocates of various departments or local government functions often peg their arguments for additional personnel to population or population growth. Although this is one way to look at staffing needs, it is not the only way—​and perhaps not even the best way—​to do so.

The problem with employee-​t o-​p opulation ratios Assessing staffing needs on the basis of employee-​to-​population ratios, as Chief Ralston proposed, is appealing for its simplicity; however, for several reasons it is a practice that usually is ill-​advised. First, the underlying rationale for using an employee-​to-​population ratio for staffing decisions is tied to a faulty premise. Its premise is the notion that the number of persons in a community signals the level of demand for a given service; however, population is usually a crude proxy for service demand. Some functions are impacted only indirectly and sometimes minimally by growth in the number of community residents. For others, the impact is more

167

168

analyzing capa city to m eet s er v ice d em a n d

direct and, with enough growth, they are likely to need more employees than in the past.1 A more important determinant of staffing needs, whether the impact of growth is direct or indirect, is the actual demand that a given population places on the service function—​for not all sets of 1,000 residents place equal demands on service providers. To demonstrate the point, simply consider the difference in the amount of police services likely to be needed for 1,000 residents in a community that has a high percentage of males in the 15-​to-​28-​year age category compared with 1,000 residents in a community that has a high percentage of affluent retirees. How well does a simple population count predict the demand for police services? Would two officers be the proper staffing for each set of 1,000 residents? Second, the use of a population ratio usually relies on a census figure representing resident population, and it totally ignores the distinctions among resident population, daytime population, and service population. The service population—​unfortunately a number that is not compiled centrally and therefore must be carefully discerned in each instance—​is only a crude indicator of demand but still a much better indicator than is the resident population or the daytime population. The service population includes residents of the city, persons who work in the city but reside elsewhere, visitors, and contractual service recipients. Nonresidents become contractual service recipients, for example, when an agreement between adjacent communities assigns responsibility for fire response in a neighborhood in one city to firefighters from the adjoining city because its station is closer to that neighborhood or when a contract with the county secures municipal recreation services for county residents, too. A city that is an employment center or a tourist destination might have a service population far in excess of its resident population. For instance, Chicago’s daytime population, which may still undercount its service population, is said to exceed its nighttime population by 750,000 people.2 Help for at least some 1 It is interesting to note that few service professionals would concede that a declining population should trigger a proportionate reduction in personnel. They might argue, in fact, that the economic conditions that brought a population decline also brought new and greater service demand, even for a smaller remaining population. 2 Michael A. Pagano, “It’s Time to Rethink How We Pay for City Services.” Crain’s Chicago Business, September 9, 2019.

s taffi n g a n a ly s i s

communities in estimating daytime population, if not service population altogether, may be available from the US Census Bureau, which provides guidance in using data from the American Community Survey to estimate daytime populations by adding nonresident employees but subtracting residents who commute to work outside the city. A link to this guidance is provided at the end of the chapter. Third, the use of a population ratio completely ignores differences in the scope and quality of services from one community to another. If the city council in one community prescribes a level of service that is much higher than in another city of similar size, it is unreasonable to expect that the staffing needs of the two communities will be identical. Fourth, the use of employee-​to-​population ratios often is disavowed by professional associations—​sometimes even by those associations assumed to have promoted the use of a ratio as a so-​called standard. Our fictional Chief Ralston, like many other police spokespersons every year, misrepresented—​ unintentionally, we presume—​police staffing averages as national standards.3 The International Association of Chiefs of Police (IACP) contends unequivocally that no staffing standard exists.4 In addition, the Federal Bureau of

3 David N. Ammons and Joshua S. Edwards, “Misrepresentation of Staffing Standards for Police.” State and Local Government Review 40, no. 3 (2008): 186–​194. 4 The IACP makes the following declaration on this subject: Ready-​made, universally applicable patrol staffing standards do not exist. Ratios, such as officers-​per-​thousand population, are totally inappropriate as a basis for staffing decisions. Accordingly, they have no place in the IACP methodology. Defining patrol staffing allocation and deployment requirements is a complex endeavor which requires consideration of an extensive series of factors and a sizable body of reliable, current data. In defining patrol staffing requirements, we consider the following factors, the mix of which is absolutely unique to each locality and agency: policing philosophy; policing priorities; police policies and practices; number of calls for service; population size and density; composition of population, particularly age structure; stability and transience of population; cultural conditions; climate, especially seasonality; policies of prosecutorial, judicial, correctional, and probation agencies; citizen demands for crime control and non-​crime control services; crime reporting practices of citizenry; municipal resources; and trends in the foregoing areas. See “Patrol Staffing and Deployment Study” (Alexandria, VA: IACP, n.d.), www.theiacp.org/​profassist/​PatrolDeployment.pdf.

169

170

analyzing capa city to m eet s er v ice d em a n d

Investigation (FBI), which reports these averages, strongly cautions against assuming that there is anything prescriptive about them.5

AVOID THIS COMMON ERROR BOX 13.2  BASING STAFFING ANALYSES ON EMPLOYEES-​ TO-​POPULATION RATIOS Some analysts assert naïvely that a department or an entire local government is efficient—​or perhaps they assert that it is understaffed—​and their analysis is based entirely on the fact that the government has a low number of employees per 1,000 population compared to other communities. Left unanswered in this superficial review are several important questions, including these: Does the population figure adequately represent demand for services? Have we ignored matters of scope and quality of services that could be equally significant in assessing efficiency or the need for staffing? Has the practice of contracting for some services been factored into this analysis?

Measure service demand directly If population is being used as a proxy for service demand and fulfills the role poorly, why not drop the proxy altogether and measure service demand directly? In the case of police, a better method would compare the number of calls for police service with the number of officers and consider whether the number of calls per officer is especially high or low relative to the ratio in other cities. If a city’s population is a high-​demand population, the number of police officers needed per 1,000 residents will be greater than in a community with a low-​demand population. Some experts even suggest disaggregating the call statistics by call type and determining the number of officers normally required for each type and the amount of time normally

5 As noted in a recent edition of the uniform crime report, Crime in the United States: 2019, the FBI declares, “[T]‌he data presented here reflect existing staffing levels and should not be interpreted as preferred officer strengths recommended by the FBI” (italics added); see https://​ucr.fbi.gov/​crime-​in-​the-​u.s/​2019/​crime-​in-​the-​u.s.-​2019/​topic-​pages/​ police-​employee-​data.

s taffi n g a n a ly s i s

Number of FTEs per MGD treated

6 WSSC scores in the third quartile (excluding Blue Plains)

5 4 3 2 1 0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Figure 13.1  Wastewater Treatment FTEs per Million Gallons Treated Daily: Washington Suburban Sanitary Commission and Peer Group of Service Providers FTE is full-​time equivalent employee. MGD is million gallons daily. Source: Veolia, Utility Benchmarking and Organizational Efficiency Review: Final Report (Boston, MA: Veolia Water North America Operating Services, LLC, 2016), 9. Report prepared for the Washington Suburban Sanitary Commission. Used by permission.

required, as a means of projecting resource needs.6 An increasing volume of some types of calls would impose greater resource pressures on the department than would a similar increase in other types of calls. Comparisons that examine staffing for the same function in multiple governments can be illuminating, if the comparisons are normalized to reflect the number of employees per unit of output. For instance, the Washington Suburban Sanitary Commission (WSSC) provides water and sewer services to nearly 1.8 million residents of Maryland’s Montgomery and Prince George’s counties, just outside Washington, DC. To assess the staffing levels of its wastewater treatment and sewer collection operations, WSSC used several measures of efficiency, including these two: “full-​time equivalent employees (FTEs) per one million gallons of wastewater treated” and “sewer collection FTEs per 100 miles of sanitary sewer.” When these ratios were compared with those of peer systems, it was clear that WSSC’s staffing level for wastewater treatments was greater than that of many of its peers (Figure 13.1) and the staffing for sewer collection fell just below the 6 “5 Myths about Police Metrics,” accessed October 11, 2017, International City/​County Management Association webinar featuring Leonard Matarese, icma.org.

171

172

analyzing capa city to m eet s er v ice d em a n d

Figure 13.2  Sewer Collection FTEs per 100 Miles of Sanitary Sewer: Washington Suburban Sanitary Commission and Peer Group of Service Providers FTE is full-​time equivalent employee. Source: Veolia, Utility Benchmarking and Organizational Efficiency Review: Final Report (Boston, MA: Veolia Water North America Operating Services, LLC, 2016), 10. Report prepared for the Washington Suburban Sanitary Commission. Used by permission.

median level (Figure 13.2). Evidence of service quality would complement these gauges of relative efficiency in a thorough staffing analysis.

Examine the evidence of understaffing directly Are presumed shortages of employees a constant problem or an intermittent problem? In the case of a chief who claims that the police department is understaffed, are all of the officers swamped all of the time or just some of the time? How often during the course of a year are all on-​duty officers engaged simultaneously in handling calls? In other words, how often is there no spare capacity in the police force? The city of Kansas City, Missouri, calls this kind of analysis blackout analysis. Blackout occurs whenever all on-​duty patrol officers are engaged on a call, which would mean that an officer on a low-​priority call would have to be pulled away if a subsequent, higher-​priority call were received. The study of one year’s evidence in Kansas City revealed 156 instances of citywide blackout, nearly all lasting less than 3 minutes.7 This confirmed to analysts that a problem existed, 7 Kansas City, Missouri, Police Department Patrol Deployment: Blackout Analysis (Kansas City, MO: City of Kansas City, City Auditor’s Office, January 1998).

s taffi n g a n a ly s i s

but it did not indicate whether the cause of the problem was a shortage of officers, as the chief contended, or suboptimal deployment of the officers already employed.

Examine the deployment of current resources Ideally, local government resources are deployed in a manner that matches the demand for services. As revealed through demand analysis (see Chapter 10), employees and other resources are sometimes deployed in a manner that only poorly matches the patterns of service demand. When this happens, a department or program can appear understaffed at some times and overstaffed at others. In the case of police patrol officers, this possibility may be examined in at least two ways. One way to examine possible deployment problems is through correlation analysis (see Chapter 6). Some police departments are especially strategic about both the design of response zones and work shifts and the assignment of officers to these zones and shifts in numbers consistent with the predictable demand for services. The success of these deployment strategies can be checked by calculating the correlation between calls for service and officers on duty, by location and time. Another way to examine possible deployment problems is through a review of instances of blackout, as revealed through blackout analysis. In Kansas City, where citywide blackout was discovered 156 times during one year and more frequently within individual response zones (Figure 13.3), analysts discovered that part of the blackout problem was attributable to questionable management practices. Some supervisors, for instance, were granting officers compensatory leave during times known to be high-​ demand periods (Saturday nights, for example), thereby increasing the likelihood of other occurrences of blackout.

Examine the consequences of presumed understaffing and consider alternative remedies Rather than debate with department managers who assert the relevance of population as a proxy for service demand, an analyst can instead request a description of the adverse consequences of this presumed understaffing. If this consequence is confirmed, the analyst can focus the analysis on options

173

174

an alyzin g capa city to m eet s er v ice d em a n d

Figure 13.3  Analysis of the Adequacy of Police Staffing and Deployment in Kansas City’s Metro Patrol Division Note: Blackout occurs whenever all on-​duty officers are engaged on calls for service. Graytime exists when all officers except one are engaged on calls. Source: Kansas City, Missouri, Police Department Patrol Deployment: Blackout Analysis (Kansas City, MO: City of Kansas City, City Auditor’s Office, January 1998), 25.

for resolving the problem—​including the possibility of increased staffing as only one of perhaps several options. Would increased staffing eliminate the adverse consequence? Would other options do so at a lower cost?

BOX 13.3  TIME & MOTION AND TIME ALLOCATION STUDIES Some local governments use rigorous observational techniques to assess current practices and to consider the need for more or fewer employees. One such technique, especially applicable in functions featuring repetitive tasks, is the traditional time-​and-​motion study. In these studies the analyst carefully observes and records each movement an employee makes in performing a given task, including the amount of time required. A perceptive analyst looks for ways to streamline the task, removing unnecessary steps and improving movements that are awkward, time-​consuming, tiring, or that put unnecessary strains on the worker. In this manner, the conscientious analyst is seeking a procedure that is both efficient and sensitive to the worker’s well-​being. When the analyst is confident that the procedures are efficient and well-​advised and that the times allotted to each movement are realistic, the movements may be combined to reveal the time required for the task

s taffi n g a n a ly s i s

to be completed by a competent worker. By combining knowledge of how many times this task will be performed in a year with the time required to perform the task, the analyst would seemingly be able to calculate the number of employees needed. However, one final adjustment is needed. Because normal fatigue affects all workers, keeping them from producing at their optimum rate throughout the workday, the analyst must add to the calculation a personal time, fatigue, and delay (PF&D) allowance, which typically ranges from 9 to 15 percent. This allowance is in addition to formal break periods. Time-​ and-​ motion studies conducted by local governments or by consultants hired by local governments can serve purposes beyond determining appropriate staff size. For example, the municipal departments responsible for reviewing property development proposals in Henderson, Nevada; San Diego, California; and Tallahassee, Florida, all use time-​and-​ motion studies to help them establish service fees that reflect the actual cost of the services being provided. In addition, San Diego uses its study to establish employee performance standards.1 A variation on the time-​and-​motion study is the time allocation study. Time allocation studies usually rely on direct observation of workers, focusing on the cataloging of tasks performed by employees daily and the amount of time consumed by each task. In this sense, these studies share some common features with work distribution analysis (Chapter 11). Although less attention is directed toward the various components of each task than is the case with traditional time-​and-​ motion studies, time allocation studies nevertheless allow the analyst to distinguish between productive and nonproductive time of workers and to analyze staffing needs. The city of Auburn, Alabama, contracted with a team of industrial engineers from Auburn University to conduct a time study of police officers to help municipal officials assess the staffing needs of the police department.2 The researchers conducted a series of ride-​alongs with different officers and catalogued and timed their activities. The data from each day were categorized as either productive work (handling dispatched and self-​initiated calls, patrolling, communications, administrative work, and court appearances), additional work (miscellaneous tasks that the analysts deemed to be of questionable value), voluntary idle time (meals, breaks, and personal hygiene time), or involuntary idle time (time spent waiting for instructions).

175

176

analyzing cap a c ity to meet s er v ice d em a nd

Among the many observations offered by the analysts following the Auburn study, two were especially influential. First, the analysts concluded that, given its current workload, the police department was understaffed by 7.4 officers, assuming a PF&D allowance of 13 percent.3 Although the ride-​alongs revealed voluntary idle time of 6 percent, this fell far short of normal PF&D expectations and confirmed the chief’s assertion that officers were being pushed to perform more work than reasonably should be expected. Second, the analysts identified tasks being performed by officers that either could have been handled by aides or officers in training (for example, directing traffic at special events) or discontinued as a police function altogether (opening locked cars for citizens as a public service). The city council authorized the hiring of four additional officers and signaled its willingness to see the elimination or reassignment of some of the low-​priority workload of officers to bring staffing in line with service demand. 1 David N. Ammons, Ryan A. Davidson, and Ryan M. Ewalt, Development Review in Local Government: Benchmarking Best Practices (Chapel Hill, NC: School of Government/​University of North Carolina and Alliance for Innovation, 2009). 2 Douglas J. Watson, Wendy L. Hassett, Jerry Davis, and Robert E. Thomas, “Use of Industrial Engineering in Measuring Police Manpower,” Public Performance & Management Review 26, No. 2 (December 2002), 132–​147. 3 PowerPoint presentation to city officials of Auburn, Alabama, presented by Jerry Davis in 2001.

Scenario: Back in Crab Harbor “You wanted to talk to me, Mayor?” Chief Ralston asked as he stepped into Carlucci’s office. “Yes, Bill. Please have a seat.” The mayor paused, then proceeded. “I spoke at length with the budget director about your request for additional officers. First of all, we don’t think that a prescriptive standard exists for the number of police officers in a community….” Chief Ralston started to protest, but he could not remember where he had first heard of the standard he had mentioned earlier or what authority to cite, so he decided not to interrupt. “… but we agree that public safety is too important to ignore the possibility that we might be understaffed there. I want you to help the budget

s taffi n g a n a ly s i s

office assess this matter. Staff in the budget office will analyze your personnel needs, but they also want to look into how you deploy your officers now and how any shortages are affecting coverage. They want to do something they called blackout analysis, so I need you to help them with that.” “Sure, I will be happy to cooperate,” Chief Ralston replied, even though he secretly wished that he could just get the resources he wanted without suffering what he considered to be an intrusion into his business. “I am sure you’d rather not have budget analysts assessing your operation and its needs,” the mayor said, “but we need to be sure we’re coming up with the right answer—​especially with a big-​ticket item like this.”

Summary advice Unfortunately, staffing analysis does not lend itself to the application of simple formulas—​even if some advocates suggest that simple population ratios will do the trick. Staffing analysis is more complicated than that and deserves the best efforts of the most perceptive analysts. In some cases, carefully conducted time-​and-​motion studies or time allocation studies may be warranted, especially where the financial and service delivery ramifications of a poor decision regarding appropriate staffing level could be particularly significant. In other cases, less elaborate studies are likely to be sufficient and more practical. In each instance, the analyst should consider the following points: •

• • •

•

The number of workers prescribed should be the number needed to perform the work when that work is properly organized and the department’s resources, including support services, are properly deployed. Staffing prescriptions should be tied to the workload to be performed. Population figures are imperfect proxies of service demand or workload; direct measures are better. Policies and strategies that affect the amount of staffing needed (for example, expected scope and quality of service, service contracting, and service automation) should be taken into account. Improper deployment of employees can masquerade as a staffing shortage.

177

178

analyzing capa city to m eet s er v ice d em a n d

•

Staffing is not an end in itself, it is a means to an end; by first identifying the desired result (for example, service improvement or the elimination of a service problem), possible staffing increases may be considered along with other options as potential means of achieving the desired end.

References Ammons, David N., and Joshua S. Edwards. “Misrepresentation of Staffing Standards for Police.” State and Local Government Review 40, no. 3 (2008): 186–​194. Ammons, David N., Ryan A. Davidson, and Ryan M. Ewalt. Development Review in Local Government: Benchmarking Best Practices. Chapel Hill, NC: School of Government/​University of North Carolina and Alliance for Innovation, 2009. International Association of Chiefs of Police. “Patrol Staffing and Deployment Study.” Alexandria, VA: IACP, n.d. www.theiacp.org/​profassist/​Patrol Deployment.pdf. City of Kansas City, Missouri. Kansas City Police Department Patrol Deployment: Blackout Analysis. Kansas City, MO: City of Kansas City, City Auditor’s Office, January 1998. Matarese, Leonard. “5 Myths about Police Metrics.” International City/​ County Management Association webinar. Washington, DC: ICMA, 2017. Accessed October 11, 2017. icma.org. Pagano, Michael A. “It’s Time to Rethink How We Pay for City Services.” Crain’s Chicago Business, September 9, 2019. www.chicagobusiness.com/​opinion/​ its-​time-​rethink-​how-​we-​pay-​city-​services. US Federal Bureau of Investigation. Crime in the United States 2019. Washington, DC: FBI, 2020. https://​ucr.fbi.gov/​crime-​in-​the-​u.s/​2019/​ crime-​in-​the-​u.s.-​2019. Watson, Douglas J., Wendy L. Hassett, Jerry Davis, and Robert E. Thomas. “Use of Industrial Engineering in Measuring Police Manpower.” Public Performance & Management Review 26, no. 2 (2002): 132–​147.

s taffi n g a n a ly s i s

Suggested for further information City of Kansas City. Kansas City Police Department Patrol Deployment: Blackout Analysis. Follow-​Up Audit. Kansas City, MO: City of Kansas City, City Auditor’s Office, September 2004. http://​webfusion.kcmo.org/​coldfusionapps/​ auditor/​listaudits.cfm.

Web resource Calculating Commuter-​Adjusted Population Estimates www.census.gov/​ topics/​employment/​commuting/​guidance/​calculations.html

179

14 STAFFING FACTOR CALCULATION PROJECTIONS FOR UNINTERRUPTIBLE SERVICES

Local government departments providing extended-​ hour services or performing functions that cannot tolerate even temporary operational lapses present staffing demands that differ from the more typical 40-​hour-​a-​week office operations. The recreation center and swimming pool may be open 70 hours a week. While the number of recreation supervisors on duty can fluctuate with absences, the lifeguard stand at the pool must never be unoccupied. The police and fire departments are around-​the-​clock operations that sometimes can tolerate a little—​but only a little—​deviation from authorized strength, but the chair of the public safety dispatcher must always be filled. Staffing decisions are complicated by the fact that many local government departments operate more hours each week than are included in the standard workweek of their employees. The recreation department, for instance, might operate 70 hours per week using employees who work 40. The police department is open for business all 168 hours of every week, but the standard workweek per employee is 40 hours. Additionally, allowance must be made for vacations, holidays, sick days, and other forms of absence.

s ta ff i n g f a c t o r c a l c ul at i o n

Calculating the staffing factor A simple technique for determining the number of employees needed to provide an extended-​hour or uninterruptible government service requires the calculation of the staffing factor, which is the number of employees needed to provide full coverage of an employee station or post. The analyst may choose to calculate employees needed for one shift or for multiple shifts, such as around-​the-​clock coverage. The first step of the two-​step calculation is as follows: E= P−A where E = the number of effective hours per employee per year or hours actually worked by the average employee P = the number of paid hours per employee per year A = the average number of hours of paid absences per employee per year (such as vacation and sick leave) Typical employees may be paid for 2,080 hours per year (40 hours a week × 52 weeks a year) but actually work only 1,856 hours if they take a two-​week vacation (2 weeks × 40 hours a week = 80 hours of vacation leave), take 10 days of paid holidays (10 days × 8 hours per day = 80 hours of holiday leave), and use 8 days of sick leave (8 days × 8 hours per day = 64 hours of sick leave). In this example, paid hours (P) are 2,080; paid absences (A) total 224 hours (80 hours of vacation leave, 80 hours of holiday leave, and 64 hours of sick leave); and effective hours (E) are the difference between the two (2,080–​224 = 1,856).1 The second step uses effective hours per employee (E) to calculate the staffing factor: Staffing factor =

Hours per year of operation E

1 Another category of paid absence from normal duties that might apply to some employees is time spent in training classes.

181

182

an alyzin g capa city to m eet s er v ice d em a n d

If, for example, the operation functions 8 hours a day, 7 days a week year-​ round (8 hours x 365 days = 2,920 hours) and a review of employee benefits or absenteeism records indicates that effective hours per employee-​ year is 1,856, then the staffing factor would be 1.57 (2,920 divided by 1,856). In other words, one full-​time and a little more than one half-​time employee—​or an equivalent combination of part-​time employees—​would be needed to provide uninterrupted coverage of the post. The following case further illustrates the calculation of the staffing factor.

FROM THE ELECTRONIC TOOLKIT BOX 14.1  CALCULATING THE STAFFING FACTOR Although there is no Excel function dedicated to staffing factor calculations, readers may find it helpful to enter the staffing factor formula into a spreadsheet and allow Excel to perform the calculations. Once the proper equation is entered into Excel, the program can repeatedly process this calculation or any other calculation with different sets of data. To set up the spreadsheet, reserve column A for data labels or symbols (for example, paid hours—​ P) and column B for data (for example, 2,080). Enter the labels and data for number of paid hours per employee per year, the average number of hours of paid absences per employee per year, and the hours per year of operation. (For directions on opening Excel and entering data refer to Box 1.1.) Strategy for populating cells in this example Row

Column A

Column B

1 2 3 4

Paid hours, P Hours of paid absence, A Hours of operation, O Staffing factor (O /​ (P–A)

2,080 224 2,920 1.57

After the data are entered, type the formula for the staffing factor into any cell of the spreadsheet using typical mathematical signs and cell addresses as the variables. If the numerical value for P (paid hours) was

s ta ffi ng f a c t o r c a l c ul at i o n

entered in cell B1, the numerical value for A (hours of paid absences) in cell B2, and the hours of operation in cell B3, enter the formula: “= (B3 /​(B1 − B2)).” Hit “enter.” Notice that the placement of the parentheses indicates which mathematical calculation to carry out first. Without the parentheses around B1 − B2, the computer would first calculate B3/​B1 and then subtract B2. Be careful to use the parentheses correctly. Once the staffing factor has been entered, the spreadsheet can be saved and the calculation can be done for other operations with different total hours or different rates of absenteeism. As an alternative to using Microsoft Excel to find the staffing factor, consider using the following online calculator: •

Shift Schedule Design at www.shift-​schedule-​design.com/​staffing%20calculator.htm

Scenario: White Dunes County, South Carolina For 13 weeks every summer, the 28,000 permanent residents of White Dunes County, South Carolina, welcome nearly 175,000 tourists a week to their oceanside communities. In past years the county’s police department has responded to the seasonal need for increased traffic control and crowd management by allowing patrol officers to work as many as 8 hours per week of overtime. However, it has become increasingly apparent to the county administrator and chief of police that the county’s 15 patrol officers are severely overworked during the peak tourist season. Therefore, they are interested in hiring supplemental, parapolice officers to provide one additional traffic control officer to each of the two busiest 8-​hour shifts 7 days a week for the 91-​day season. Fortunately, there is an abundant supply of qualified persons to hold the seasonal positions. How many persons will need to be hired as temporary officers in order to provide 16-​hour supplemental coverage for 91 days?

183

184

analyzing capa city to m eet s er v ice d em a n d

Applying the staffing factor to White Dunes County The first step in answering this question (and in calculating the staffing factor) is to determine the number of effective hours that could be provided by each supplemental employee per year, confined in this instance to a 91-​day period. If each employee is scheduled and paid for 40 hours a week for 13 weeks, the number of paid hours (P) is 520. Full-​time, year-​round employees of White Dunes County are granted paid holidays, vacation time, sick days, and other forms of paid leave, but the benefits of seasonal employees are much more limited. After consulting with police departments in neighboring counties that already use seasonal police personnel, the White Dunes County police chief recommended that each parapolice officer hired for the summer receive up to 3 compensated days off—​2 allowable sick days and 1 paid holiday. Projecting that all three days of paid absences would be taken (A = 3 days × 8 hours = 24 hours), the chief calculated effective hours per employee (E) as follows: E= P−A = 520 − 24 = 496 The calculation of the staffing factor could address a single shift or both shifts simultaneously. A single-​shift staffing factor is as follows: hours per year of operation E 8 hours per day × 91days = 496 728 = 496 = 1.47

Staffing factor =

One full-​time and one half-​time employee, or an equivalent combination of part-​time employees, should be sufficient to provide uninterrupted coverage of the post. A consolidated staffing factor for 16-​hour per day coverage could be derived by doubling the single-​shift staffing factor

s ta ff i n g f a c t o r c a l c ul at i o n

or by inserting a comprehensive figure for hours of operation into the calculation: hours per year of operation E 16 hours per day × 91days = 496 1456 = 496 = 2.94

Staffing factor =

Three full-​ time employees, or an equivalent combination of full-​ and part-​time employees, should be sufficient to provide uninterrupted coverage of the post.

Using the information Determining the staffing factor of a service is a major element in calculating the costs of adding personnel, but other elements are also important. The costs of employee benefits as well as additional equipment, supplies, work space, or other facilities must also be considered. In the White Dunes County case, for example, known costs related to wages, uniforms, insurance, and other expenses should provide a reasonable estimate of the costs for the three parapolice officers needed to cover the extra positions. If desired, the estimate for supplemental staffing on one or both shifts can be compared with other staffing alternatives (such as continuing to allow full-​time, year-​round patrol officers to work overtime or contracting out for additional police or security officers) to determine which approach to providing seasonal police protection for the county will cost the least.

Utility of the staffing factor Calculating a staffing factor can quickly enlighten a debate over the need for and cost of putting another police officer or firefighter on duty around the clock. Too often advocates of such a move naïvely assume that the cost is roughly equivalent to one annual salary. A well-​prepared public official can quickly project a more realistic estimate of the cost of such a proposal.

185

186

analyzing capa city to m eet s er v ice d em a n d

BOX 14.2  OTHER APPLICATIONS OF STAFFING FACTOR CALCULATIONS Staffing factor calculations are relevant to any local government operation that has posts that must never be left unoccupied during operating hours. A recreation position solely responsible for supervising an after-​ school or weekend playground cannot be left unoccupied just because the employee filling it is on vacation. Similarly, a library position solely responsible for covering the children’s section during the evening hours cannot be left unoccupied when the assigned employee is out sick. In these and countless other cases involving a variety of departments, full coverage during a prescribed set of hours is required.

Suggested for further information Kelley, Joseph T. Costing Government Services: A Guide for Decision Making. Washington, DC: Government Finance Officers Association, 1984. See “Applications in Practice,” 55–​72.

Web resources City and County of San Francisco (CA), “Methodologies for Determining Staffing Requirements” https://​sfbos.org/​22-​methodologies-​determining​staffing-​requirements For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

Part IV WHAT DOES IT COST?

What could be simpler than identifying costs? If you want to know the cost of an item, just look at the price tag, right? For the cost of a local government function, just check the bottom line of the department’s budget, correct? Unfortunately, it is not that simple. An item’s price tag will tell you its price but not its cost—​at least not the full cost of owning and operating the item over its useful life. The bottom line of a department’s or a program’s budget will perhaps tell you its anticipated direct cost, but this presumed bottom line is likely to omit overhead costs reported elsewhere in the budget. Knowing how to count all the costs and how to count them fairly is important for gauging a program’s efficiency and for setting fees at cost-​recovery levels. A careful approach to identifying and counting costs—​including knowing which cost elements to include and which to exclude in a given type of analysis—​is important for privatization decisions, life-​cycle costing, deciding whether to lease equipment or buy it, and for accurate cost-​effectiveness analysis. In determining costs, we may also need to consider the financing costs associated with a purchase made over a period of time. How much will be paid—​or saved—​in interest by choosing one financing option rather than another? How much must be set aside each year at the projected rate of interest in order to accumulate the amount of resources needed to pay in

188

what does it cos t?

advance for a desired capital project? How can we take into account analytically the net advantage or net disadvantage of choosing one option for the use of resources over another option—​in other words, one opportunity over another? These questions can be answered readily with a few simple calculations. Suppose we want to compare today’s costs with the costs of some previous time? What mathematical adjustments will allow us to remove the effects of inflation so we can more fairly compare the value of current revenues or expenditures with the value of revenues or expenditures at some point in the past? This, too, can be answered simply, as explained on the pages that follow.

15 ADJUSTING FOR INFLATION WHEN COMPARING REVENUES OR EXPENDITURES

Comparing the amount of revenues received by a local government one year with the amount received in another year can be misleading. Although the more recent number may be larger, it might represent resources with less “buying power” than the smaller figure from an earlier year. Similarly, steadily increasing expenditures might be more attributable to inflation than to service expansion or loss of efficiency. Meaningful comparisons across years are possible only if the figures are adjusted to reflect “constant dollars.” This is what the city of Golden, Colorado, was doing, for example, when it displayed its expenditures per household and per capita adjusted for inflation (Figure 15.1).

Scenario: Keystone, Nevada The city manager of Keystone, Nevada, was already worried about next year’s budget, even though the budget hearings were still three months away. Years ago in college a professor had described what he called

190

what does it cos t?

Figure 15.1  Expenditures Per Household and Per Capita by the City of Golden, Colorado (Adjusted for Inflation) Source: City of Golden, Colorado. Biennial Budget: 2019–​2020, 288.

“rational-​comprehensive decision making” and its application to the budget process, but very little seemed rational about the decision making taking place in Keystone these days. Keystone had the third-​highest rate of population growth in the state, and the citizens were demanding improved services. With city council elections coming next spring, incumbents were already bracing for the onslaught of challengers grumbling about runaway expenditures. Just this morning a coalition of council members sent word through the mayor that it expected the city manager to hold the line on expenditures and would not, under any circumstances, consider a property tax increase for next year. “Mayor, how can you and the others preempt me like this?” asked the city manager. “We’re addressing important community needs, and we’re managing our resources wisely. I had hoped to get a fair audience for what I am sure will be a solid budget proposal. Why the panic? Our increases haven’t been extravagant.” “How can you say that?” responded the mayor. “Municipal expenditures are up 13 percent in two years’ time. In some major departments—​like the police department—​it’s been even higher. Don’t think for a minute our opponents haven’t pointed that out to potential supporters.” The mayor and the council members now banding together were clearly worried about their public image and the upcoming election. The mayor had been elected on the strength of a campaign that emphasized her

a d j us t i n g f o r i n f l at i o n

business experience and her fiscally conservative philosophy. No wonder she felt stung by the 13 percent increase! Soon after the mayor left, the city manager called the budget officer to his office. After describing the substance and tone of the mayor’s message, the city manager said, “It won’t be business-​as-​usual this year. There’s no way that we’ll be allowed to tap resources the way we have in the past few years. The best we can hope for is a modest increase in expenditures funded by tax base expansion rather than a tax rate increase. But we won’t even get that if we aren’t prepared.” The city manager laid out a strategy for meeting the “runaway expenditures” charge head-​on. He directed the budget officer to prepare a comparison of expenditures for this year and the previous two years using “constant dollars” rather than “current dollars.” “Start with our highest-​priority items, like police patrol. Let’s see if we can get some information out that will help us—​and the mayor and council—​defend what we’ve been doing.”

Inflation indexes When shoppers grumble, “A dollar doesn’t go as far as it used to,” they are right. An item that costs $1.10 today may have cost $1.00 only a few years ago. Inflation diminishes the value of money. Four quarters and a dime in that shopper’s hand might look like $1.10 (and it is $1.10 in current dollars), but it is only $1.00 in constant dollars if these coins can buy no more than what a dollar could buy in the base year. Converting current dollars to constant dollars is a simple matter. Various price indexes are available for that purpose, the most popular one being the Consumer Price Index (CPI), compiled by the Bureau of Labor Statistics (see Table 15.1).1 The CPI measures changes in a variety of consumer products and is used frequently in labor negotiations and as a guide to various cost-​of-​living adjustments. The version of the CPI most widely used is the Consumer Price Index for Urban Consumers or CPI-​U. The index’s rate of change from one year to another—​for example, from 251.107 in

1 For CPI information online, see the US Bureau of Labor Statistics website, www.bls.gov/​ cpi/​.

191

192

what does it cos t?

Table 15.1  Selecting an Inflation Index: CPI versus IPD, 1990–​2019 Year

2019 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 2001 2000 1999 1998 1997 1996 1995 1994 1993 1992 1991 1990

Consumer Price Index for all urban consumers

Implicit Price Deflators for state & local government consumption expenditures and gross investment

Average annual CPI

Percentage change from previous year

State & local Implicit Price Deflator

Percentage change from previous year

255.657 251.107 245.12 240.007 237.017 236.736 232.957 229.594 224.939 218.056 214.537 215.303 207.342 201.6 195.3 188.9 184 179.9 177.1 172.2 166.6 163 160.5 156.9 152.4 148.2 144.5 140.3 136.2 130.7

1.81 2.44 2.13 1.26 0.12 1.62 1.46 2.07 3.16 1.64 -​0.36 3.84 2.85 3.23 3.39 2.66 2.28 1.58 2.85 3.36 2.21 1.56 2.29 2.95 2.83 2.56 2.99 3.01 4.21

114.969 112.775 108.450 105.770 105.598 105.645 103.279 100.000 97.739 94.669 92.048 92.558 88.133 83.617 79.609 75.369 72.05 69.815 68.281 66.032 63.008 60.63 59.471 58.177 56.871 55.394 54.002 52.69 50.953 49.153

1.95 3.99 2.53 0.16 -​0.04 2.29 3.28 2.31 3.24 2.85 -​0.55 5.02 5.40 5.03 5.63 4.61 3.20 2.25 3.41 4.80 3.92 1.95 2.22 2.30 2.67 2.58 2.49 3.41 3.66

Source: US Department of Labor, Bureau of Labor Statistics, “Consumer Price Index,” August 28, 2020 (www.bls.gov/​cpi/​data.htm). US Department of Commerce, Bureau of Economic Analysis, “Table 1.1.9 Implicit Price Deflators for Gross Domestic Product,” September 30, 2020 (www.bea. gov/​tools/​). Select “Interactive Data,” then go to “National Data GDP & Personal Income;” choose “Selected NIPA Tables.” Find “Table 1.1.9.” The state and local government IPD is a single row toward the bottom of this table.

a d j us t i n g f o r i n f l at i o n CPI-urban

State and local price deflator

6

Inflation rate from prior year

5 4 3 2 1 0 -1

1991

1994

1997

2000

2003

2006

2009

2012

2015

2018

Figure 15.2  Tracking Differences in CPI and IPD Inflation Rates Note: Calculated from index values in Table 15.1

2018 to 255.657 in 2019, a rise of 1.81 percent—​reflects inflation in consumer prices. A second index—​one more tailored to the needs of local government—​ is the Implicit Price Deflator (IPD) for state and local government expenditures, which is compiled by the US Department of Commerce’s Bureau of Economic Analysis and is also shown in Table 15.1.2 The IPD is a better choice for most local government purposes because it reports an index based specifically on the kinds of goods and services that state and local governments purchase. Even so, the IPD, like all other indexes, is an imperfect gauge of inflation, and users should be aware of that. How similar are the results of the CPI and the IPD? The two indices generally track each other closely (Figure 15.2). In the last three decades, the variance in a given year usually has been less than 1 percent. The notable exception was in the early 2000s when the state and local price deflator

2 For IPD information online, see the US Bureau of Economic Analysis website, www. bea.gov/​national/​nipaweb/​index.asp. Go to “Selected NIPA Tables,” choose “Table 1.1.9 Implicit Price Deflators,” select “Annual Series.” The state and local government IPD is a single row toward the bottom of this table.

193

194

what does it cos t?

had increases larger than the CPI for multiple years. Much of this was due to increases in the price of oil, which affected a number of items purchased by state and local governments, such as asphalt and fuel, more significantly than the market basket items of general consumers. A third index relevant to local governments is the Municipal Cost Index (MCI), produced by and appearing in American City & County magazine (Table 15.2).3 The MCI draws upon the Consumer Price Index, the Producer Price Index, construction cost indexes, and independently compiled data in an effort to reflect changes in the cost of providing municipal services. In addition to these three indices of broad inflation are indices for particular subsets of the economy. The US Bureau of Labor Statistics compiles price indexes for different regions and for different goods such as gasoline or food. The bureau’s Producer Price Indexes focus on items such as building construction, road construction, and transportation. An analyst wishing to examine expenditure increases in street repaving, for instance, might turn to one of these more specialized gauges of inflation. Converting current dollars to constant dollars using any of these indexes involves simple mathematics. The formula for doing so is: base year indexx* current index* = current revenues or expenditures in base year dollars

current dollar revenues or expenditures x

* The index chosen could be the Consumer Price Index (CPI), Implicit Price Deflator (IPD), or Municipal Cost Index (MCI)

First, a base year of the analyst’s choosing is selected. Then current dollars are multiplied by the ratio of the base-​year index (CPI, IPD, or MCI) to the current year of the same index. The resulting figure will be the buying power of current dollars in base-​year dollars. The term current is used for convenience; it may refer to this year or to any year other than the selected base year by simply saying “current dollars in 2019” or whatever year the analyst chooses. 3 For MCI information online, see http://​americancityandcounty.com/​mciarchive/​ #Archive.

a d j u s ti n g f o r i n f l at i o n

Table 15.2  Municipal Cost Index, 1999–​2020 Date

Municipal Cost Index

Percentage change from previous year

January 2020 January 2019 January 2018 January 2017 January 2016 January 2015 January 2014 January 2013 January 2012 January 2011 January 2010 January 2009 January 2008 January 2007 January 2006 January 2005 January 2004 January 2003 January 2002 January 2001 January 2000 January 1999

255.56 251.43 246.71 239.46 231.46 233.5 230.6 227.4 223.8 215.1 208.4 206.1 201.3 192.4 188.4 179.2 167.9 163.0 159.3 158.5 153.5 149.3

1.6 1.9 3.0 3.5 −0.9 1.3 1.4 1.6 4.0 3.2 1.1 2.4 4.6 2.1 5.1 6.7 3.0 2.3 0.5 3.3 2.8

Source: Data drawn from American City & County, “Municipal Cost Index.” Accessed August 28, 2020. www.americancityandcounty.com/​municipal-​cost-​index/​. Republished by permission. Copyright 1999–​2020 by American City & County/​Informa.

BOX 15.1  ACCESS TO INFLATION INDEXES The latest Consumer Price Index information and an online “inflation calculator” are available at the website of the US Bureau of Labor Statistics (www.bls.gov/​cpi/​home.htm). On-​line access to Implicit Price Deflators is available through the US Department of Commerce’s Bureau of Economic Analysis (www.bea.gov/​ data/​prices-​inflation/​gdp-​price-​deflator). Go to “Selected NIPA Tables;” choose “Table 1.1.9 Implicit Price Deflators;” select “Annual Series.” The state and local government IPD is a single row toward the bottom of this table.

195

196

what does it cos t?

Access to the Municipal Cost Index is available through the website of American City & County magazine: www.americancityandcounty.com/​ municipal-​cost-​index/​.

Scenario: Converting Keystone’s police patrol expenditures to constant dollars The budget officer in Keystone compiled a brief summary of police patrol positions and expenditures for the three most recent fiscal years (see Table 15.3). The increase in expenditures for the police department was slightly greater than the increase for the city as a whole—​14.4 percent, rather than 13.1 percent. The increase was in part attributable to adding two patrol officers and three public service officers during the last two years. Those increases, of course, had nothing to do with inflation. But what would the increase have been had it not been for inflation?

AVOID THIS COMMON ERROR BOX 15.2  ACCOUNTING FOR CHANGES IN INFLATION OR POPULATION CHANGE BY ADDITION OR SUBTRACTION An analyst attempting to find the rate of inflation or population growth for a multiyear period must not simply add single-​year rates together, because this would fail to take account of compounding. For example, an increase of 3.1 percent in inflation or population one year followed by 4.2 percent the next year does not mean that the two-​year rate of growth has been 7.3 percent (3.1 + 4.2). Consider a city with 100,000 population that grows to 103,100 one year (3.1 percent increase) and from 103,100 to 107,430 (4.2 percent increase) the next. The two-​year population gain from 100,000 to 107,430 residents is a gain of 7.43 percent—​not 7.3 percent. Calculating multiyear changes in percentages is not simply a matter of adding or subtracting single-​year rates. Do the math to find the actual multiyear percentage!

a d j us t i n g f o r i n f l at i o n

Table 15.3  Police Patrol: Positions and Expenditures, City of Keystone Fiscal year 2017 Positions Sergeants Patrol officers Public service officers Total Expenditures Salaries and benefits ($) Supplies ($) Other services ($) Maintenance ($) Capital ($) Total ($)

Fiscal year 2018

Fiscal year 2019

4 26 2 32

4 27 4 35

4 28 5 37

1,602,085 87,473 89,393 58,830 202,070 2,039,851

1,794,125 91,276 88,044 64,655 171,058 2,209,158

1,946,313 94,058 41,292 68,445 183,482 2,333,590

The budget officer converted each of the expenditure figures in Table 15.3 to constant dollars using the IPD for state and local governments. She decided to use the IPD because it is based on the things local governments buy, plus she liked being able to mention the Bureau of Economic Analysis as she explained her calculations. Keystone’s fiscal year begins July 1 of the calendar year and ends June 30 of the following year. Because these fiscal years do not coincide precisely with the calendar years listed in the IPD table, the budget officer had to make a reasonable IPD selection that could be applied uniformly across her data. For example, the choice for a given fiscal year could be the IPD for the calendar year in which the fiscal year began, the year in which it ended, the first quarter IPD, or perhaps the midpoint between two annual IPDs. For the sake of simplicity, she decided to use the IPD for the year in which a given fiscal year ended. For example, fiscal year 2017–​2018 (FY 2018) ended June 30, 2018, so for FY 2018 she used the 2018 IPD. The 2017 IPD of 108.45 was used as the base year, and the unadjusted FY 2017 expenditures were regarded as 2017 dollars. The calculations for converting total expenditures for FY 2018 and FY 2019 to 2017 constant dollars for the patrol activity of the Keystone cops are shown in Table 15.4. The full set of expenditures converted to 2017 dollars is shown in Table 15.5. “This will help, but we still have a lot of work to do,” remarked the city manager when the budget officer showed him the constant dollar figures.

197

198

what does it cos t?

Table 15.4  Converting Keystone Police Patrol Expenditures for Fiscal Year 2018 and Fiscal Year 2019 to 2017 Constant Dollars currentdollar revenues base year IPD × = current revenues or expenditures in base yeardollars or expenditures current IPD Total expenditures for FY 2018 converted to 2017 constant dollars $2,209,158 ×

108.45 = $2,124, 435 112.775

Total expenditures for FY 2019 converted to 2017 constant dollars $2,333,590 ×

108.45 = $2,201,270 114.969

Notes: For the FY 2018 calculation, $ 2,209,158 is the total FY 2018 expenditure reported in Table 15.3. The IPDs of 108.45 and 112.775 for 2017 and 2018, respectively, are reported in Table 15.1. For the FY 2019 calculation, $ 2,333,590 is the total FY 2019 expenditure reported in Table 15.3. The IPDs of 108.45 and 114.969 for 2017 and 2019, respectively, are reported in Table 15.1.

Table 15.5  Constant Dollar Comparison of Police Patrol Expenditures, City of Keystone, 2017–​2019

Salaries and benefits Supplies Other services Maintenance Capital Total Percentage increase from FY 2017

FY 2017 expenditures ($)

FY 2018 expenditures in 2017 dollars

FY 2019 expenditures in 2017 dollars

1,602,085 87,473 89,393 58,830 202,070 2,039,851 –​

1,725,319 87,776 84,667 62,175 164,498 2,124,435 4.1%

1,835,953 88,725 38,951 64,564 173,078 2,201,270 7.9%

Note: All figures for fiscal years 2018 and 2019 have been converted to 2017 constant dollars using implicit price deflators for state and local government consumption expenditures and gross investment.

“We’re still up 7.9 percent in constant dollars in just two years’ time, but that’s a lot better than having to explain a 14.4 percent jump.” “We’ve increased the number of positions in that division by 16 percent to try to improve services and meet the demands of a growing population,” commented the budget officer.

a d j us t i n g f o r i n f l at i o n

BOX 15.3  OTHER APPLICATIONS OF INFLATION ADJUSTMENTS Opportunities for the application of inflation indexes in local government are numerous. Such indexes are often used to peg cost-​of-​living increases in employee wages, annual adjustments in multiyear service contracts, and multiyear revenue and expenditure comparisons. Adjusting figures to constant dollars in such comparisons permits a more reasonable evaluation of fiscal trends.

“That’s true,” responded the city manager. “Let’s try to attack this thing on two fronts. See if you can come up with some performance indicators that show improved service quality to help us defend expenditures in excess of inflation.4 Also, let’s adjust our expenditure figures to account for a growing population as well as inflation. Get the best population figures you can from the planning department and convert those constant dollar numbers to expenditures per capita reported in constant dollars. With the growth we’ve experienced in the last two years, I think we might even see a declining constant dollar expenditure per capita! When we make those adjustments and compile our improved performance indicators, I’m confident we will present a very defendable record.”

Suggested for further information Leazes, Francis J., Jr., and Carol W. Lewis. “Now You See It, Now You Don’t.” In Casebook in Public Budgeting and Financial Management, edited by Carol W. Lewis and A. Grayson Walker III, 189–​194. Englewood Cliffs, NJ: Prentice-​ Hall, 1984.

Web resources American City & County, “Municipal Cost Index” www.americancityandcounty. com/​municipal-​cost-​index/​ Bureau of Economic Analysis, “GDP Price Deflator” www.bea.gov/​data/​prices-​ inflation/​gdp-​price-​deflator 4 See Chapter 3 “Performance measurement and monitoring.”

199

200

what does it cos t?

US Bureau of Labor Statistics, “Consumer Price Index” www.bls.gov/​cpi/​ home.htm US Bureau of Labor Statistics, “CPI Inflation Calculator” www.bls.gov/​data/​ inflation_​calculator.htm Also, consider a web search using these key words: consumer price index, implicit price deflator, and adjusting for inflation. For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

16 BASIC INVESTMENT CALCULATIONS FIGURING INTEREST RATES AND YIELDS

Calculating interest rates and projecting investment yields are fundamental tasks that come up time and again—​especially in the analysis of proposals having long-​term financial ramifications for a local government. Getting it right is important.

Scenario: Denia, Ohio Manfred D. “Manny” Garber, budget analyst for the city of Denia, Ohio, shook his head as he reviewed the city’s fleet inventory and recent vehicle replacement patterns. “This is crazy! Do you realize what we are doing? Each year we make our equipment replacement decisions mostly on the basis of how tight the budget is. The age and condition of our vehicles and other equipment is a relatively minor consideration. If we are short of cash, we expect our mechanics to perform miracles. We just

202

what does it cos t?

keep the clunkers another year and ignore the fact that we probably are not saving a dime by trying to stretch the service life of some of these money-​pits.” “Sure, it might be shortsighted compared to a rational equipment management strategy,” replied Al Shorty, Denia’s finance director, “but in the 11th hour when we are trying to balance the budget, I wouldn’t want to be the one telling the city council that we cannot afford to hire a few additional police officers, that we need to trim the employee pay increase by a percentage point or two, or that we need to raise taxes because a lot of our fleet is pretty decrepit. When it comes to choices like these, it’s a pretty easy call. Let’s get out the bailing wire and patch Old Daisy up for another year of work.” “Even if it costs us more to keep Old Daisy running than we avoid in prorated replacement costs?” This was the latest in a series of conversations on the topic of vehicle replacement strategies that had taken place intermittently in the city manager’s office, the finance department, and fleet maintenance. Everyone was in agreement that it would be nice to have an equipment replacement fund, with annual contributions in the form of “rent” from vehicle users in amounts sufficient to allow timely replacement. With a fund dedicated to equipment replacement, the only haggling would be over whether a given vehicle was really worn out and whether the replacement specifications were appropriate—​not whether equipment purchases would unbalance the budget. But getting there from here was the problem. How do you come up with the money to establish the fund in the first place? The last comment on the topic by the city manager was hardly reassuring: “The next time the city gets a revenue windfall, establishing an equipment replacement fund will be high on my list of priorities.” “Great,” Manny remembers thinking, “and when will that ever happen?” Patience was not one of Manny’s strengths, but pragmatism was. “Can’t we at least start building a fund rather than waiting for a windfall? Let’s put a little something into the fund each year, draw interest, and let it grow until it reaches the point that we can begin operating from it. Getting where we want to be in 5 or 10 years is better than not getting there at all.” “Put together a proposal,” Al replied, “and we’ll see. Maybe it will fly.”

ba s ic i n v e s t m e n t c a l c ul at i o n s

Investment calculations 101 Analysts working with investments or attempting to evaluate alternate investment strategies would be wise to include four fundamental investment formulas in their toolkit of analytic techniques. These formulas will allow them to calculate: 1. yields based on simple interest 2. yields based on intrayear compound interest (for example, compounded quarterly) 3. the interest rate on an investment, knowing the initial investment, length of term, and yield 4. the yield from a series of even, annual cash investments (that is, annuities) The fourth calculation will be especially pertinent to Manny’s proposal for the city of Denia, but let us take them up one at a time.

Calculating yields based on simple interest FVn = PV (1 + i)

n

where FVn = the future value of an investment following n periods PV = amount of initial investment i = interest rate per period n = the number of interest periods The entry (1 + i)n may be calculated by hand or may be found in a table of “Future value interest factors” (see Appendix C). Using this formula, an analyst could calculate, for instance, the return from $10,000, invested three years at 7 percent annual interest. FV3 = $10,000 × (1 + .07) = $10,000 × 1.2250 = $12,250 3

203

204

what does it cos t?

Calculating yields based on intrayear compound interest  i  tn FVn = PV 1 +   t where t is the number of times per year that interest is compounded. Using this formula, an analyst could calculate, for instance, the return from $10,000, invested two years at 8 percent interest compounded quarterly.  .08  (4)(2) FV2 = $10,000 1 + 4   = ($10,000) (1 + .02) = $10,000 × 1.717 8

= $11,717 Once again, the future value interest factors table (Appendix C) may be consulted in this calculation, but this time the formula calls for the interest rate to be divided by the number of payments per year (8 percent divided by 4 = 2 percent) and the number of years to be multiplied by the number of payments per year (2 × 4 = 8). The cell entry for 2 percent and 8 periods is 1.1717. Comparing this result with the yield on simple interest demonstrates the impact of compounding.1

Calculating the interest rate on an investment If analysts know the amount of initial investment, length of term, and yield, they can calculate the interest rate using the following formula: I=

SP − PP 360 x PP D

1 The future value interest factor (FVIF) for this example was 1.1717, compared with a FVIF of 1.1664 for two years at 8 percent simple interest. On an investment of $10,000, that is a difference in yield of $53.

ba s ic i n v e s t m e n t c a l c ul at i o n s

where I = interest rate SP = investment’s value at sale or redemption (sale price) PP = initial investment (purchase price) D = number of days investment was held Using this formula, an analyst could calculate, for instance, the interest rate on a 90-​day US Treasury bill, purchased for $9,850 and having a face value of $10,000, as $10,000 − 9,850 360 × $9,850 90 = 0.01523 × 4 = 6.09%

I=

Calculating the yield from a series of even, annual cash investments If a local government puts away equal annual sums (that is, annuities) in order to accumulate funds for some future use, an analyst wishing to compute the total of those funds at a future date may do so using the following formula: SN = A ( SAFi,n ) where SN = sum of the annuity after n payments A = amount deposited annually SAFi,n = sum of annuity factor (see table in Appendix D) Using this formula, an analyst would know that a city or county that plans to deposit $20,000 annually for five years at 7 percent interest will have a sum of $115,014 at the end of that period.2 SN = A ( SAFi,n ) = $20,000 × 5.7507 = $115,014 2 Note that the SAF table in Appendix D assumes that annual payments are made at the end of each year. The SAF for every interest rate in period 1 is 1.0000, indicating that no interest is earned until the second period.

205

206

wh at doe s it cos t?

Alternatively, the analyst may calculate the annual payment amount necessary to accumulate a desired sum in a given timeframe using the following formula: A=

SN SAFi,n

FROM THE ELECTRONIC TOOLKIT BOX 16.1  INTEREST CALCULATIONS Interest calculations can be tricky when done by hand because there are exponents and fractions to deal with. Microsoft Excel allows the user to eliminate these obstacles and simply plug numbers into a command that outputs the yield or interest rate. To calculate yields based on simple interest, use the Future Value—​ FV—​function. After entering an equals sign (=) and the FV command (FV), the user must enter several variables to complete the calculation. In the parentheses after the function, the user must enter: • • • • •

the interest rate the number of periods the payments made during each period the initial amount (or present value) and the type of payment made

Figure 16.A  Future Value Function in Excel

ba s ic i n v e s t m e n t c a l c ul at i o n s

For the example used in this chapter, the initial investment was $10,000 with an interest rate of 7 percent over three years. To enter this example in the Excel command, the user would input “=FV(.07,3,0,10000,1),” and the output would be the future value of $12,250. Notice that because no payments are made over time, the third variable is “0.” Also, the final variable “1” indicates that the payment was made at the beginning of the pay period. Excel users may find it especially helpful—​and simpler—​ to choose “Formulas” from the menu options at the top of the screen, and select the “FV” function from the financial function options. This method includes prompts for the information needed in the parentheses. Alternatively use the “Fx” option on the toolbar and search for the “FV” function by name. Finding the yield from intrayear compound interest is a bit more complicated. Before using the FVSCHEDULE function, the user must create the schedule of interest rates to be compounded. By dividing the interest rate by the number of times compounded a year, the user can begin to create the schedule. If the interest rate is 8 percent and it is compounded quarterly, then the user should enter “=.08/​ 4” in cell A1. The computer would output “.02.” Because the investment encompasses a two-​year period, the number of times it will be compounded is 8. Highlight cell A1, and move the mouse to the lower, right-​hand corner of the cell. When the black cross appears, drag the mouse down to cell A8 or to the total number of times the investment will be compounded. Now that the compounding schedule is created, it is time to use the FVSCHEDULE function. After the command, the variables entered in the parentheses by the user should indicate the initial investment and the cells that contain the schedule. In the example above, the user would enter “=FVSCHEDULE(10000,A1:A8),” and the computer would calculate the yield. (The future value in this case is $11,717.) Excel also has a command that computes the interest rate on an investment, but instead of knowing the number of days the investment is held, the user must know the actual dates in order to use the command. To find an interest rate, use the INTRATE function. In the parentheses after the command, indicate: • • •

the date the investment began the date the investment ended the initial investment

207

208

wh at doe s it cos t?

• •

the sale price and the type of day count to use

If a US Treasury bill was purchased on January 1, 2020 for $9,950, had a face value of $10,000, and was held for 90 days, or until April 1, 2020, the interest rate may be calculated by entering “=INTRATE(“1/​1/​ 2020”, “4/​1/​2020”, 9950, 10000, 2).” It is important to remember that quotation marks are needed around the dates. Also, the 2 appearing at the end of the sequence tells the computer to use a normal day count in which the number of days is divided by 360. (The interest rate in this case is 1.99 percent.) Other options are available for the day count, which may seem unusual to those new to financial calculations. Consult the link below for explanations about different options for the day count in financial functions. Calculating the yield from a series of even, annual cash investments on Excel can be done with the same function used to calculate simple interest. Use the FV function (future value) and enter the same data variables in parentheses after the command: the interest rate, the number of periods, the payments made during each period, the initial amount, and when the payment is made. To find the total value for the example in this chapter, the user would enter “=FV(.07,5,20000,0,0)”, and the computer would report the sum of the annuity after five payments—​i.e., $115,014. Notice that the principal amount is zero and the type of payment is also zero, indicating the payments are not made in the beginning of the period. Link to explanation for how days are counted in financial functions: https://​en.wikipedia.org/​wiki/​Day_​count_​convention

FROM THE ELECTRONIC TOOLKIT BOX 16.2  CALCULATING PAYMENT AMOUNTS FOR A LOAN You know how much is being borrowed. You also know the length of the loan and rate of interest. So, how much will the periodic payments need to be? Excel has a payment function (PMT) for calculating the periodic payments needed for a loan, assuming uniform payments and a constant

ba s ic i n v e s t m e n t c a l c ul at i o n s

rate of interest. If your spreadsheet lists the interest rate, the number of payment periods, the loan’s principal or present value, and perhaps also the future value and whether payments will be made at the beginning or end of each period, you can easily find the payment amount. Just select an empty cell, type “=” and click on the fx that appears above the spreadsheet. An “insert function” box will appear; select PMT. Now there will be prompts calling for each of the needed entries. These entries can be referenced from the spreadsheet by clicking the cell on the sheet. This function requires the user to enter the interest rate in decimal form (for example, 6 percent is .06), the number of periods (for example, the number of annual payments or the number of monthly payments), the loan principal or its present value, the future value (this is optional), and whether payments are made at end of the period (enter “0” or leave it blank which is the default) or at the beginning of the period (“1”). Alternatively, you can just select a blank cell enter “=PMT(Interest Rate, Number of Periods, Present Value, Future Value (optional),Type of Payment-​end or beginning of period)” and type the values directly into the function. However, spreadsheet best practice is to reference cells containing the needed data. To skip the Future Value entry, just enter a comma in its place. The PMT function requires a single constant interest rate and a constant payment schedule for the length of the loan. If payments will be made annually, the specified interest rate will be the annual rate of interest and the number of periods will be the number of years of the loan. If payments will be made monthly, the specified interest rate will be the monthly rate of interest and the number of periods will be the number of months of the loan. So, a 20-​year loan with an interest rate of 6 percent and annual payments would list the interest rate as .06 and the number of periods as 20. But if the payments were to be made monthly, the number of periods would be 240 (20 years x 12 months per year) and the interest rate would be the monthly interest rate rather than the annual rate. Simply dividing the annual interest rate by 12 would provide an approximation of the monthly interest rate, but a more precise calculation would be better. The future value in the Excel function is an optional argument and is only needed if after the last payment there will be a remaining cash balance, which is not the case for most loan repayments. The Type of Payment specifies whether the payment will be made at the end of the period (enter “0” or leave it blank) or at the beginning of the period (enter “1”).

209

210

wh at doe s it cos t?

With appropriate entries in the PMT function, Excel will calculate the payment amount. Note that if the principal or present value is put in as a positive number, the result from the payment calculation will be a negative number reflecting an outflow or expense. Entering the principal as a negative number, reflecting a liability, will produce a positive payment from the function.

Back in Denia Manny pored over the fleet management and purchasing records pertaining to the maintenance and acquisition of the city’s vehicles. He had the advantage of hindsight in determining when vehicles should have been replaced. By calculating annual depreciation (see Chapter 18) and plotting the rise in maintenance costs as vehicles moved through their optimum service periods, he could identify the point at which replacement would have made the most sense. Next, Manny devised a vehicle replacement schedule that prescribed replacement for vehicles in a given category based on miles or hours of service, but allowed exceptions for vehicles experiencing unusually high or low maintenance expenses. Applying this schedule to the current fleet, Manny found that systematic replacement could result in capital outlays as high as $850,000 in some years and as low as $380,000 in others. The average year appeared to require about $500,000 in capital expenditures for vehicles.

AVOID THIS COMMON ERROR BOX 16.3  ASSUMING A 20% GAIN FOLLOWED BY A 20% LOSS… …puts you back where you started. It does not. Test it out by starting with $1,000 and first increasing that amount by 20% ($1,000 + 200 = $1,200), then reducing the new total by 20% ($1,200 − 240 = $960). Sadly, you lost money and you lose with any percentage—​not just 20%.

ba s ic i nv e s tm e nt c a l c ul at i o n s

Furthermore, you lose whether the gain comes first in the sequence or the loss comes first. So what’s the lesson? Realize that it takes a bigger percentage gain to neutralize a given percentage loss…and do the calculation every time.

To have a chance for his plan to succeed, Manny needed a strategy that would provide seed money for the proposed equipment replacement fund. For the time being, the city would continue to make equipment acquisition decisions as part of the annual budget deliberation, while setting aside a small appropriation each year that could draw interest and eventually grow into a fund large enough to do the job. He thought a fund of $550,000 would be adequate. The formula for annuities required Manny to estimate an interest rate for the city’s investments. He decided to estimate an interest rate of 8 percent and inserted it into the formula. After trying several combinations of payment amounts and payment periods, Manny found a combination he liked, one calling for seven years of payments: A= =

SN SAFi,n $550,000 = $61,640 8.9228

Using this calculation, Manny decided to recommend seven annual appropriations of $61,640 into a newly established equipment replacement fund. At the end of seven years, the fund would have $550,000—​more than the average annual equipment replacement requirement but not enough to cover a peak purchasing year. Beginning in the eighth year, “rental fees” for equipment use would be assessed to departments, and equipment purchases from the fund would commence. These rental fees would cover annual maintenance expenses plus depreciation. In the eighth year, the special appropriations from the general fund would cease, and purchases would be made from the equipment replacement fund, replenished annually by rental fees. Until then, the city would continue to make vehicle purchases from available funds through the annual budget process.

211

212

what does it cos t?

“I like this, Manny,” Al said when he saw the proposal. “I think we have a decent chance to get approval of an appropriation this size. But what happens if that first year of purchases from the fund turns out to be one of the big years? It couldn’t handle $850,000.” “That would be a problem, all right, but we are just as likely to get one of the lighter years. If it does turn out to be a peak purchase year, we have two things going for us. First, there is a little cushion built into the fund itself. Second, we will be drawing rental fees from the departments. That should give us a little flexibility, as long as we don’t get two big years in a row. What do you think our chances would be for a special appropriation to the fund if it’s a huge year for vehicle purchases?” “I suppose that is a possibility, but I would say the smarter choice would be to manage our fleet between now and then to be sure that we do not hit a peak purchasing year just as the equipment replacement fund swings into action.”

BOX 16.4  BASIS POINTS Financial institutions realize that even small fractions of percentage points can make huge differences in costs or gains in major transactions. Accordingly, they divide 1 percent into 100 parts, called basis points. One basis point is 0.01 percentage point; 50 basis points is 0.50 percentage point or one-​half of 1 percent. Local government finance departments sometimes use basis points in setting targets and reporting their success in investing idle cash. Because the economy affects the actual rate of return, these departments peg their target to a conservative instrument that moves with the economy. For instance, they might set as their target beating the yield on 90-​day US Treasury bills or beating the Treasury bill rate by at least 20 basis points. Then at the end of the year they report their average rate of return not only as a percentage but also as the number of basis points above or below the 90-​day Treasury-​bill rate that they achieved.

ba s ic i n v e s t m e n t c a l c ul at i o n s

Suggested for further information Adams, Andrew, Philip Booth, David Bowie, and Della Freeth. Investment Mathematics. Hoboken, NJ: John Wiley & Sons, 2003. Coe, Charles. Governmental and Nonprofit Financial Management. Vienna, VA: Management Concepts, 2007. Lovelock, David, Marilou Mendel, and A. Larry Wright. An Introduction to the Mathematics of Money: Saving and Investing. New York: Springer Science and Business Media, 2007. Ross, Sheldon M. An Elementary Introduction to Mathematical Finance, 3rd ed. Cambridge: Cambridge University Press, 2011. The Economist Numbers Guide: The Essentials of Business Numeracy, 6th ed. New York: Public Affairs, 2014. Thomsett, Michael C. The Mathematics of Investing: A Complete Reference. New York: John Wiley & Sons, 1989. Wang, Xiaohu. Financial Management in the Public Sector: Tools, Applications and Cases. Armonk, NY: M.E. Sharpe, 2006.

Web resources For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

213

17 THE TIME VALUE OF MONEY OPPORTUNITY COSTS, DISCOUNTING, COMPOUNDING, FUTURE VALUE, AND PRESENT VALUE

A financial advice columnist once explained the concept of opportunity costs by describing the choice a car buyer faces when choosing between a Toyota Camry and a BMW Series 2.1 In today’s prices, the choice of the Camry would save $13,000, which if invested in stocks might realistically grow to $33,719 in 10 years—​about the time the Camry would be approaching 150,000 miles and perhaps need to be replaced. If the same choice was repeated on the replacement vehicle, the nest egg at the end of the second car’s life could have grown to more than $87,458. Now, making the columnist’s point about opportunity costs, would driving the BMW rather than the Camry really be worth more than $87,000 to you? This is the opportunity cost associated with the initial decision. A decision by public officials to spend financial resources eliminates the possibility of investing them for future use. A decision to use a local government’s resources for one purpose eliminates the possibility of using those same 1 Douglas R. Sease, “You Can’t Take Car Envy to the Bank,” Wall Street Journal, Sunday, April 20, 2008.

th e t i m e va l ue of m o n e y

resources for other purposes. Snap judgments on the wisdom of such a decision—​choosing to spend rather than invest or to apply resources to one opportunity rather than another—​are often subjective and typically reflect the biases of program proponents and opponents. Those who favor the expenditure are sure it will be money well spent; those opposed are just as convinced that it is wasteful. In many cases, a more objective assessment is possible. The wisdom of a resource allocation decision can be assessed by placing the decision in context and by considering the time value of money. Was it a good decision compared with some other option—​including the option of investing these funds? What opportunities did the local government forgo when it opted as it did? A standard method of making such an assessment is to compare the “return” or benefit from a project with the return on a conservative investment of an equal amount of funds.2 On that basis, the use of funds for a program with substantial revenue-​generating or cost-​avoidance potential would seem to have an advantage over a conservative investment of an equal amount of funds, which in turn would seem to have an advantage over non-​revenue-​generating alternatives; but the time value of money comes into play in different ways. A dollar in hand today is considered to be more valuable than a projected dollar to be received in the future for at least three reasons: 1. It is a sure thing—​the risk of nonreceipt is eliminated. 2. The possibility of consumption today rather than tomorrow is highly prized (if owners must wait before gaining access to their resources, compensation in the form of interest payment is expected in return for deferred gratification). 3. Inflation erodes the buying power of money.3

2 Stacking proposed uses up against possible investment returns is an analytic approach that is clearly relevant in cases where idle funds are available for either project expenditure or investment, but it is also relevant in cases where project funds would have to be raised through a special levy. If the voters reject the tax levy, the jurisdiction, of course, would not have the opportunity to invest these funds; however, taxpayers would have that opportunity themselves through individual, personal investments. In such instances, the question becomes: Would the citizens be better off allocating their resources to this project or investing the money elsewhere? 3 A. John Vogt and Samuel H. Owen Jr., “Cost Calculations for Lease-​ Purchase Decisions: Methods,” in A Guide to Municipal Leasing, ed. A. John Vogt and Lisa A. Cole (Chicago: Municipal Finance Officers Association, 1983), 116–​117.

215

216

what does it cos t?

Put a dollar bill under your mattress tonight, then pull it out in four or five years and you will see the point clearly. Because your dollar has gained no interest under your mattress, it will not have kept pace with rising prices and will not buy as much as it would have if spent today. You would have to have something more than one dollar in four or five years in order to possess as much value as your dollar has today. For purposes of analysis, equivalent values for the present and future need to be calculated—​in other words, adjustments must be made for the time value of money. A “compounding” factor can be used to calculate the “future value” of a given quantity of current resources. Think of future value as principal plus interest. Alternatively, a “discounting” factor can be used to calculate the “present value” of a given quantity of future resources to see how much buying power in today’s dollars it will have at that point in the future. The compounding factor will be: (1 + i)n where i = interest rate n = the number of interest periods The formula for calculating the future value of a given quantity of current resources is: F + P (1 + i)n where F = future resources P = present resources i = interest rate n = the number of interest periods Alternatively, a given quantity of future resources may be discounted to reveal its present value by using the following formula:

 1  P =F n  (1 + i )  where P would reflect the amount of current dollars possessing value equivalent to the specified amount of future dollars.

th e t i m e va l ue of m o n e y

Scenario: South Fork, Texas Trey Dixon, the director of parks and recreation for the town of South Fork, Texas, had hired two college interns, Larry Pendergrass and Emile Busby, to assist with various analytic projects. One of the interns was plainly bewildered. “I guess I screwed up this analysis the director wanted,” Larry confided. “I worked up some cost and revenue figures for the swimming pool proposal and showed him that we would break even on our investment in 17 years and finally start to have overall net revenues for the project at that point, but he wasn’t impressed (see Table 17.1). He kept talking about ‘opportunity costs’ and ‘present value’ and asked me to recalculate the numbers. I was too embarrassed to tell him that I didn’t know what he was talking about. Do you have any idea what he meant?” “I think so,” Emile replied. “It all has to do with choices about the use of resources and the different benefits for each choice. When I pick one option, I lose the opportunity for the other benefits—​hence the name, ‘opportunity costs.’ I seem to recall that most analysts calculate opportunity costs Table 17.1  Estimated Revenues and Costs for the Swimming Pool Proposal Year

Revenues

Costs

Net revenues ($)

Cumulative net revenues ($)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Total

$0 0 220,000 250,000 280,000 310,000 340,000 370,000 400,000 430,000 460,000 490,000 520,000 550,000 580,000 610,000 640,000 $6,450,000

$450,000 1,250,000 190,000 210,000 210,000 230,000 250,000 270,000 290,000 310,000 330,000 350,000 370,000 390,000 410,000 430,000 450,000 $6,390,000

−450,000 −1,250,000 30,000 40,000 70,000 80,000 90,000 100,000 110,000 120,000 130,000 140,000 150,000 160,000 170,000 180,000 190,000 $60,000

−450,000 −1,700,000 −1,670,000 −1,630,000 −1,560,000 −1,480,000 −1,390,000 −1,290,000 −1,180,000 −1,060,000 −930,000 −790,000 −640,000 −480,000 −310,000 −130,000 60,000

217

218

what does it cos t?

conservatively as the difference between the value of using the money now versus investing it, adjusted for the difference between what a dollar is worth today and what a dollar will be worth in a few years, since investment is one of the options I could pick.” “You’ve got to be kidding, Emile! You haven’t invested a dime in your life. You are Mr. Immediate Gratification, himself.” “This is theoretical, Larry. Theoretically, I could invest a dollar and earn interest, or I could spend it and have the benefit of whatever I buy. The difference between the cumulative value of something I buy today and the cumulative value of a safe investment is the opportunity cost. You can figure the present value of a dollar available sometime in the future by discounting it for the effect of inflation between now and then. I really don’t think it’s too tough to do.” “What does all of that have to do with my projection of costs and revenues for a new outdoor swimming pool equipped with slides and water features?” “It’s a multiyear projection, and the director doesn’t want the changing value of money to distort the analysis.”

Discount factor Calculating the present value of a future dollar is not particularly difficult, but it does require the analyst to exercise judgment. The calculation depends on an estimate of the rate of interest that could be earned on a safe investment (for instance, government bonds or certificates of deposit) by the local government over the period of time in question or by the jurisdiction’s taxpayers if the money is left in their hands. That projected interest rate, of course, is related to the analyst’s expectations regarding inflation.4

4 In choosing a particular rate of interest for this calculation, the analyst may wish to consider current interest rates on the government’s investments, forecasts of leading economists, and the rate of interest the government pays on its municipal bonds. At best, however, the choice will probably involve some guesswork. Ideally, the rate selected will prove accurate in the long run, but minimally it should be deemed reasonable at the time by most consumers of the analysis.

th e t i m e va l ue of m o n e y

Once an interest rate is projected, a discount factor for each year of the analysis can be calculated using the formula noted previously: Discount factor =

1 (1 + i)n

where i is the interest rate and n is the number of years. By multiplying a future dollar amount by the relevant discount factor, its present value may be determined.

Present value calculations Revising the swimming pool analysis proved to be surprisingly simple—​ and the results were surprising as well. Larry projected an interest rate of 6 percent and applied discount factors based on that estimate to his earlier analysis (see Tables 17.1 and 17.2). Table 17.2  Present Value Analysis of the Swimming Pool Proposal Year

Revenues ($)

Costs ($)

Net revenues ($)

Discount factora (i = 6%)

Present value of net revenuesb ($)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Total

0 0 220,000 250,000 280,000 310,000 340,000 370,000 400,000 430,000 460,000 490,000 520,000 550,000 580,000 610,000 640,000 $6,450,000

450,000 1,250,000 190,000 210,000 210,000 230,000 250,000 270,000 290,000 310,000 330,000 350,000 370,000 390,000 410,000 430,000 450,000 $6,390,000

−450,000 −1,250,000 30,000 40,000 70,000 80,000 90,000 100,000 110,000 120,000 130,000 140,000 150,000 160,000 170,000 180,000 190,000 $60,000

0.9434 0.8900 0.8396 0.7921 0.7473 0.7050 0.6651 0.6274 0.5919 0.5584 0.5268 0.4970 0.4688 0.4423 0.4173 0.3936 0.3714

−424,530 −1,112,500 25,188 31,684 52,311 56,400 59,859 62,740 65,109 67,008 68,484 69,580 70,320 70,768 70,941 70,848 70,566 −$625,224

a Discount factor = 1/​(1+i)n where i is the interest rate (6 percent in this case) and n is the number of years. In this example, for instance, the discount factor for the sixth year is 1/​(1+.06)6 or 0.7050. b The present value is calculated by multiplying net revenue by the discount factor.

219

220

wh at doe s it cos t?

The proposed pool would be equipped with several slides and other water features in hopes of drawing large enough crowds to generate considerable revenue. Proponents hoped there would be sufficient operating revenues from admissions and concessions to more than offset operating costs and eventually recover capital costs. Property and design costs were estimated at $450,000. Projected costs of the pool, slides, and related facilities came to $1,250,000. Land acquisition and facility design would occur the first year; construction would occur the second. Annual operating costs of $190,000 were projected in the third year—​the first year of operation—​ followed by a gradual escalation to $450,000 in year 17. Revenues from pool admissions and concessions would not begin until the pool opened the third year, and they were projected by Larry at a modest level in the beginning but growing fairly rapidly with the projected development of new residential subdivisions in the vicinity.

FROM THE ELECTRONIC TOOLKIT BOX 17.1  FINDING PRESENT VALUE AND THE DISCOUNT FACTOR For present value and discount factor calculations, users may find it beneficial to insert their formula into a spreadsheet. First, enter the variable names or symbols (for example, “future resources” or “F=”) in consecutive cells of a given column. Then, enter the appropriate numerical values into cells adjacent to the labels or symbols in the next column. (For directions on opening Excel and entering data refer to Box 1.1.) To calculate present value, the variables that must be entered are future resources, interest rate, and the number of interest periods. After the data are entered, type the equation into any cell using the names of the cells in which the data appear rather than the symbols. For example, if future resources were entered in cell B1, the interest rate in cell B2, and the number of interest periods in cell B3, enter “=(B1*(1/​ ((1+B2)^B3)))” to calculate present value. Notice the importance of the placement of parentheses. To calculate the discount factor, simply enter the appropriate equation. If using the same data as in the present value calculation, enter “=(1/​ ((1+B2)^B3))” to find the discount factor. It is important to remember to

t he ti m e va l ue o f m o n e y

use an equal sign when entering an equation in the spreadsheet. If the equal sign is forgotten, the computer will read the equation as text and will not complete the calculation. Excel has functions for both the Present Value and the Future Value. The PV or Present Value function in Excel may be used to calculate present value. This function returns the present value of an investment. It requires the user to enter information about the interest rate, number of periods, amount of payment per period, future value, and whether the payments are made at the beginning or end of the period. Enter “=PV(interest rate, number of periods, payment, future value, type of payment)”. This returns the present value of the series of future payments. Excel’s function for calculating Future Value has a similar structure. It requires the user to enter information about the interest rate, number of periods, regular payments, the present value, and whether payments are made at the beginning or end of the period. To use it enter =FV(interest rate, number of periods, payment, present value, type of payment). Both the PV and FV functions in Excel require a single interest rate and a constant payment. For more complicated assumptions with changing interest rates or changing payments, the user should construct a spreadsheet using the formulas as described earlier in this box.

The new table prepared by Larry once again showed deficits the first two years with no revenues coming in. But with the pool opening for operation in the third year, net revenues are anticipated each year. Although the application of discount factors yields present value deficits less substantial than raw figures for the first two years, their application in later years has an even greater effect on net revenues. As a result, the grand total net revenue of +$60,000 based on raw dollar tabulation is shown to have a present value that is actually a deficit of −$625,224 or nearly 10 percent of total capital and project costs through year 17. The problem is that the high-​cost years come at the beginning of the project and therefore are discounted very little, while the high-​revenue years come late in the period of analysis when discounts are more substantial. “I’m really disappointed,” muttered Larry. “I never expected the pool to be enough of a moneymaker to pay back the full capital expense in 10 years’ time, but I thought a 17-​year project would be a winner—​and

221

222

what does it cos t?

therefore not too tough of a sell. I still think we need that pool even with a net cost of more than $600,000, but this present-​value analysis is a real eye-​opener.”

Utility of opportunity cost analysis An articulate proponent can sometimes mesmerize public officials into believing that a favored program or facility costing $1 million to establish and generating $100,000 per year in revenues will pay for itself in 10 years’ time. The argument’s arithmetic is simple; unfortunately, its logic is simplistic. The calculation of opportunity costs would reveal a substantial deficit even after the 10th year. The project could still be a good idea, but public officials should make their decision with full awareness that it would not have paid for itself in the declared time period.

BOX 17.2  OTHER APPLICATIONS OF PRESENT VALUE ANALYSIS Present value analysis is likely to produce revealing insights in many local government situations, including the following: • • • •

when a proposal requires a major up-​ front expenditure that proponents promise “will pay for itself” over a period of several years when user fees projected for a multiyear period are established to recover major capital costs incurred at the outset of the fee period when a grant provides start-​up costs but requires the local government to continue the program into the future when the local government is choosing between two projects and wishes to compare their net present values before making the choice

References Sease, Douglas R. “You Can’t Take Car Envy to the Bank.” Wall Street Journal, Sunday, April 20, 2008. Vogt, A. John, and Samuel H. Owen Jr. “Cost Calculations for Lease-​Purchase Decisions: Methods.” In A Guide to Municipal Leasing, edited by A. John

th e t i m e va l ue of m o n e y

Vogt and Lisa A. Cole, 115–​ 145. Chicago: Municipal Finance Officers Association, 1983.

Suggested for further information Bandy, Gary. Financial Management and Accounting in the Public Sector, 2nd ed. New York: Routledge, 2015. See Chapter 5, “Costing and Project Appraisal,” pp. 177–​203. Eller, Warren S., Brian J. Gerber, and Scott E. Robinson. Public Administration Research Methods: Tools for Evaluation and Evidence-​Based Practice, 2nd ed. New York: Routledge, 2018. See “Applied Decision Tools,” Chapter 19, especially 402–​408. Granof, Michael H., Saleha B. Khumawala, Thad D. Calabrese, and Daniel L. Smith. Government and Not-​for-​Profit Accounting: Concepts and Practices, 8th ed. Hoboken, NJ: Wiley, 2019. Michel, R. Gregory. Cost Analysis and Activity-​Based Costing for Government. Chicago: Government Finance Officers Association, 2004, 70–​77. —​—​—​. Decision Tools for Budgetary Analysis. Chicago: Government Finance Officers Association, 2001. The Economist Numbers Guide: The Essentials of Business Numeracy, 6th ed. New York: Public Affairs, 2014. See “Finance and Investment,” 39–​62. Wang, Xiaohu. Financial Management in the Public Sector: Tools, Applications and Cases. Armonk, NY: M.E. Sharpe, 2006.

Web resources https://​financeformulas.net/​Future_​Value.html www.calculator.net/​future-​value-​calculator.html www.treasury.govt.nz/​sites/​default/​files/​2017-​06/​twp17-​02.pdf www.wbdg.org/​FFC/​FED/​OMB/​OMB-​Circular-​A94.pdf Also, consider a web search using these key words: present value, future value, appropriate discount rate. For a template and exercise associated with this chapter, see https://​toolsfor decisionmaking.sog.unc.edu.

223

18 SIMPLE OPTIONS FOR ANNUALIZING COSTS OF CAPITAL ITEMS USAGE-​RATE AND STRAIGHT-​LINE DEPRECIATION

Local government officials attempting to analyze program costs often are tempted to look only at the bottom line of a department’s budget or expenditure record. Unfortunately, the bottom line sometimes provides an inaccurate reflection of annual expenses. Some relevant program costs may be reported elsewhere. Furthermore, the expenses shown in a given year may be influenced dramatically by the peaks and valleys of capital expenditures. The cost of facility construction, equipment acquisition, and other capital items should be included when considering total program costs—​but including capital costs in their entirety whenever they occur would distort an analysis. A more reasonable approach would be to depreciate capital items and include for each year a portion of the capital costs proportionate to the useful life of an item.

a n n u a liz in g c o s t s of c a p i ta l i t e m s

Scenario: Horace and Jasper, Iowa Nat Kilgos had been on the job in the research and budget department of the city of Horace, Iowa, only a few days when he was given the assignment of comparing the animal control program in Horace with that of nearby Jasper. “See if we ought to make any changes to our system or if we should adopt Jasper’s approach to animal control,” the research and budget director had said. Nat plunged eagerly into the task, diligently collecting program descriptions and budget documents, interviewing program supervisors of each operation, and pressing for performance measures and other indicators of the level of service and success of each program. He was determined to prove his worth as an administrative assistant. The services provided by the animal control programs in the two communities were strikingly similar. The numbers of animals handled by each program annually were almost identical, and both programs boasted very favorable qualitative performance measures. The most apparent operating difference was that Horace’s program was managed and staffed by city employees, while Jasper’s was provided through a municipal contract with a local animal protection group. That group used the city-​owned animal shelter and charged the city of Jasper a fixed annual fee to cover its expenses in running the program, which included the cost of full-​and part-​time employees, operation and maintenance of the shelter, utilities, supplies, vehicles, and other equipment. Nat could see little difference in quality or scope of services to recommend one program over the other. Furthermore, his comparison of budget documents showed a difference of less than $10,000 between the two programs. The contractual program had the lesser expenditure, but there was no guarantee that a contractual system would work as well in Horace as it apparently did in Jasper. Should Horace give contracting for this service a try? Nat did not think so. All in all, an incentive of less than $10,000 seemed insufficient to justify the gamble and the program disruption that would be caused by such a substantial operating change. Prior to finalizing the report for the research and budget director, Nat took his tabulation (see Table 18.1) to Chris Bogle, a senior budget analyst who had offered to review his work. “I just don’t see that it would make sense to make the change for such a modest potential payoff,” explained Nat.

225

226

what does it cos t?

Table 18.1  Animal Control Annual Budget Expenditures Expenditures

City of Horace ($)

City of Jasper ($)

Salaries and benefits Supplies Contractual/​other services Maintenance Capital Total

181,207 13,626 15,870 2,730 0 $213,433

–​ –​ 204,000 –​ –​ $204,000

“You might be right,” replied Chris after spending a few minutes reviewing the tabulation and several supporting documents, “but I see a few holes in your analysis that you’ll want to plug before presenting your report. Using budgets is handy for comparing expenditures, but you have to be careful to be sure that you’re including everything.” “I did that,” responded Nat. “The contract in Jasper includes everything except the animal shelter building and land, and that’s not a cost factor in the Horace budget either, because it was paid for long ago. And the personnel costs for Horace include fringe benefits and everything. It looks like a fair comparison to me.”1 “What about the cost of administering the contract?” “It’s negligible. I checked with the city manager. But I guess I should put a little something in there to reflect the fact that it’s not absolutely cost free.” “What about our equipment costs? This year’s budget shows nothing for equipment acquisition just because we replaced both trucks during the last two years and replaced the other major equipment three years ago. It 1 The administrative assistant is touching on a key point of relevance to comparisons such as this. If the purpose of the analysis is to determine which entity operates most efficiently, then all entities in a comparison should be examined on an equal basis (see Chapter 19). Including the fringe benefits or building depreciation for one and not the other would distort the comparison. In the case of Horace and Jasper, fringe benefits have been included for both programs. The animal shelter and associated land for both programs are city owned, so excluding depreciation or facility rental for both entities simplifies the analysis with little impact on results. However, if the Jasper animal shelter had been privately owned and provided as part of the contract, Horace’s calculation would have required adjustment to see which of the two was the more efficient operation. If, on the other hand, the purpose of the analysis is to consider contracting for a service, only the costs that would be eliminated with the shift of operations should be included in the Horace total (see Chapter 20).

a n n u a liz in g c o s t s of c a p i ta l i t e m s

wouldn’t be fair to throw the full acquisition price into the annual cost comparison, but it’s not fair to simply ignore it just because we didn’t buy anything this year. I’ll bet the Jasper contract includes an amount being set aside by the contractor for equipment replacement. For a fair comparison, you ought to depreciate our capital equipment and show the annual depreciation in your analysis.”

Annualizing capital expenses Frequently the costs of a capital item are borne at one point in time, while the benefits are spread over several years. A truck purchased in 2018, for example, might still be providing benefits to its owner in 2025. Two of the simplest methods of allocating reasonable costs of a capital item over its useful life are allocations based on usage rate and allocations based on straight-​line depreciation. The rationale for allocating costs according to usage rate is tied to the premise that the deterioration of some capital items is caused more by the use of those items than by the passage of time. A machine that can be expected to last through 10,000 hours of operation—​whether those hours occur in 2 years’ time or 10 years’ time—​is a good example. If the cost of such a machine is $10,500 and it is expected to have a salvage value of $500, it would be reasonable to assign annual costs for that equipment at the rate of $1 per hour of operation. The annual cost would be dependent on the anticipated or actual usage of the equipment per year. The formula for usage rate allocation of cost is: ai =

ui (C − S ) U

where ai = capital expense allocation for time period i ui = usage units consumed during time period i U = total estimated usage units in the life of the asset C = cost of the asset S = salvage value after U usage units

227

228

what does it cos t?

FROM THE ELECTRONIC TOOLKIT BOX 18.1  FINDING USAGE-​RATE ALLOCATION OF COST Even without a specific Excel function for calculating usage rate allocation of cost, computers nevertheless can be programmed to perform the calculation of this or any of the other formulas mentioned in this book. After the formula is programmed into Excel once, the usage rate allocation can be determined again and again with different data. First, enter the variable names or symbols—​usage units consumed, total estimated usage units in the life of the asset, cost of the asset, and salvage value—​in four consecutive cells in a given column. (For directions on opening Excel and entering data, see Box 1.1.) In the next column enter the numerical value for each variable in the cell adjacent to a given label or symbol. For example, you might enter “u(i)=” in cell A1 as a prompt for the numerical value of “2000” in cell B1. Then, in any cell, enter the formula for usage rate allocation of cost using the cell names as the variables. For example, if the numerical variables were entered in order from B1 through B4, enter the formula: = (B1/B2)*(B3–B4) where the cells are as follows: B1 = Usage units consumed B2 = Total estimated usage units in the life of the asset B3 = Cost of the asset B4 = Salvage value and after pressing “enter” the usage rate allocation of cost will appear. The calculation can be done for different sets of data by simply changing the data in cells B1 through B4.

An alternative, more suitable for allocating annual cost for capital items whose deterioration is perhaps as much due to the passage of time as to actual usage, is straight-​line depreciation. For example, although many buildings deteriorate more quickly with heavy use, they nevertheless deteriorate with the passage of time even with fairly light use. Even when

a n n u a liz in g c o s t s of c a p i ta l i t e m s

deterioration is directly related to usage, straight-​line depreciation offers the advantage of minimizing guesswork in projecting annual costs. The analyst, for example, will no longer need to forecast the number of hours the item will be used or the number of visitors that a facility will host in the coming year.

FROM THE ELECTRONIC TOOLKIT BOX 18.2  STRAIGHT-​LINE DEPRECIATION Calculating straight-​line depreciation using Excel is simple because of the SLN function. Click on the cell where you want the annual depreciation amount to appear. Then type “=SLN(cost,salvage,life)” to initiate the function. In the parentheses, enter the initial cost, the salvage value, and the total number of time periods in the item’s expected life. For example, the straight-​line depreciation of an item with the initial cost of $10,500, a salvage value of $500, and an expected life of five years would be found by entering “=SLN(10500,500,5)”. Hit “enter” and the outcome of $2,000 will appear in the designated cell. As an alternative to Excel, consider online calculators: www.calculatorsoup.com/​calculators/​financial/​depreciation-​straight-​ line.php www.calculator.net/​depreciation-​calculator.html

The acquisition costs for most cars and trucks could be annualized using either usage-​rate or straight-​line depreciation. For the usage-​rate approach we could estimate the usage units (miles of use) in the vehicle’s life—​ perhaps 90,000 miles. Or for the straight-​line approach we could estimate the vehicle’s years of life. If it gets an average of 15,000 to 20,000 miles of use per year, we could project five years of life. For the sake of simplicity, straight-​line depreciation might be chosen as the allocation technique, using the following formula: ai =

C−S N

229

230

what does it cos t?

where ai = capital expense allocation to each time period C = cost of the asset N = total number of time periods in the item’s expected life S = salvage value after N periods

Making appropriate adjustments Two days later, the administrative assistant returned to the senior budget analyst with a revised set of figures. Following further inquiry into Jasper’s cost for administering its contract, Nat was prepared to defend a cost estimate of $5,000 for contract administration. The program costs for Jasper’s animal control program were adjusted upward by that amount. The adjustment of Horace’s numbers to account for capital costs on an annual basis was considerably greater (see Horace’s calculation below). The two extended-​bed pickup trucks used by the animal control program each cost $26,500 and, according to the superintendent of the city garage, had an expected life of three years and a projected salvage value of $1,000. All other capital equipment associated with the animal control program (for example, fiberglass beds with animal enclosures, mobile radios, nets, catch poles, and strobe lights) had been replaced in a package purchase three years before at a total cost of $24,500. That equipment has a projected life of 10 years and an anticipated salvage value of $500. The application of the straight-​line depreciation technique indicates an annualized capital cost of $19,400 ($17,000 for the two trucks and $2,400 for the other equipment) (see below). ai =

C−S N

Pickup trucks: $26,500 − 1000  = $8,500 per year per truck  3  = $8,500 × 2 trucks Total   = £17,000 per year  $17,000 trucks  Other equipment: 2, 400 other  $24,500 − 500  $19, 400 = $2, 400 per year  10

a n n u a liz i ng c o s t s o f c a p i ta l i t em s

Table 18.2  Animal Control: Revised Annual Costs Expenditure

City of Horace ($)

City of Jasper ($)

Salaries and benefits Supplies Contractual/​other services Maintenance Capital Contract administration Total

181,207 13,626 15,870 2,730 19,400 –​ $232,833

–​ –​ 204,000 –​ –​ 5,000 $209,000

Adding $5,000 for contract administration to Jasper’s program costs and $19,400 in annualized capital costs to Horace’s total stretched the previous cost difference of $9,433 to a new estimated difference of $23,833 or about 10 percent of current costs (see Table 18.2). “I’m still not sure that the possibility of saving $23,000 or $24,000 is worth the risk of disrupting a good program,” worried Nat. “I don’t even know if we have an animal welfare group or other contractor willing to tackle the job for an amount similar to what Jasper pays.” “I share your concern,” said Chris, “but with your latest revisions, the numbers are becoming a lot more tempting.”

BOX 18.3  TAKING A SECOND LOOK: SENSITIVITY ANALYSIS Many analytic techniques require an analyst to make a few assumptions or offer a prediction or two. Once the assumption or prediction is entered into the formula or model, the calculations proceed and the analysis can be performed. But what if the assumptions or predictions are not quite on target? Would a small error dramatically change the analytic result or just affect it a little bit? In other words, how sensitive is the analysis to the assumptions embedded within it? The testing of this sensitivity is called sensitivity analysis, a practice described in Chapter 7. Here is how sensitivity analysis could be applied to the work of budget analyst Nat Kilgos. Nat’s analysis showed that the city of Horace might be able to save almost $24,000 per year by contracting out its animal control operation, as its neighboring city does. The cost comparison that led to this conclusion hinged on the accurate depiction of Horace’s annualized capital costs.

231

232

what does it cos t?

Nat had calculated annual capital costs for Horace’s animal control operation at $19,400, based on straight-​line depreciation and the garage superintendent’s declaration that the animal control pickups had an expected life of three years. This morning, Nat began to wonder about the superintendent’s figure. His wondering led to some investigating, and by late afternoon he stepped into the office of Senior Budget Analyst Chris Bogle with a new piece of information for their analysis. “I think we need to make it really clear to the city manager and city council that the potential savings figure we are giving them with our analysis is probably the most we could hope for, based on our annualized costs and our hope for favorable contract costs,” Nat said. “In fact, I think our annualized costs might be depicted a little high in our analysis, contributing a few thousand extra dollars to the projected savings.” Nat explained that he had been thinking about whether a three-​year life expectancy for the animal control pickups was realistic, and as a result he had done some checking on the Internet. He found that the manufacturers of current pickup models were forecasting greater durability and a longer useful life. He also checked the city’s own replacement records and found that only a few pickups were replaced at three years. Even heavy-​duty pickups usually were in service for at least four years and most for five years. “I recalculated the annualized capital costs for pickups, using four-​, five-​, and six-​year life expectancies to compare with the three-​year life expectancy we projected earlier,” Nat explained. “Take a look at this table.” Nat’s sensitivity analysis (Table 18.3) showed that the projected savings fell by 17 percent when a four-​year expected life was used in the calculation; by 28 percent with a five-​year expected life; and by 35 percent with a six-​year expected life. “I don’t think that projecting a six-​year life for these pickups is reasonable, even though we have some in the fleet that have lasted that long—​and even longer. But I think that projecting a five-​year life is more realistic than forecasting a three-​year life.” “It makes you stop and think,” Chris replied, as he studied the table. “Maybe we should adjust our analysis.” “I think we should,” Nat responded. “We have a good in-​house animal control program right now, and I would hate to scrap it for an expected

a nn u a liz ing c o s t s o f c a p i ta l i t e m s

Table 18.3  Projected Cost Savings: Sensitivity to Expected Life of Capital Equipment Optional life expectancies of animal control pickups

Salvage value of trucks Annualized capital cost for two pickupsa Annualized capital cost for other equipment Total projected capital costs Projected savings from contractb

3 Years

4 Years

5 Years

6 Years

$1,000 $17,000

$800 $12,850

$500 $10,400

$200 $8,767

$2,400

$2,400

$2,400

$2,400

$19,400 $23,833

$15,250 $19,683

$12,800 $17,233

$11,167 $15,600

a See Horace’s calculation showing 3-​year life expectancy. Adjusting the denominator yields the results appearing in this row. b See calculation in Table 18.2 showing Horace’s cost, assuming a capital expenditure total of $19,400. The total cost exceeds that of Jasper by $23,833. Replacing the capital expenditure entry in Table 18.2 by the amounts shown here yields the savings shown in this row.

$24,000 savings if the actual savings turns out to be less. If the savings are only going to be $17,000, I think the city council will think twice about moving away from a proven program to take a risk on something else.”

Utility of techniques for annualizing the cost of capital items A persistent problem in the analysis of local government operations is the tendency to understate departmental or program expenditures. Sometimes this is caused by accounting or budgeting practices that fail to assign program costs precisely, perhaps allowing overhead costs to be absorbed elsewhere in the budget or reporting employee benefits in a separate account. Still another common cause of underreporting is failure to allocate capital costs properly. Many local governments include the entire costs for all capital acquisitions in the year in which they are purchased. This practice not only inflates program expenses in the year in which a major capital outlay happens to occur, but it also understates the true costs of the program in the subsequent years when the costs for these items are invisible. Annualized

233

234

what does it cos t?

capital costs contribute to a much more reasonable statement of annual program costs.

BOX 18.4  OTHER APPLICATIONS OF DEPRECIATION CALCULATIONS Annualizing capital costs through usage-​ rate or straight-​ line depreciation is useful when comparing a local government’s costs to those of other governments or to the bids of private vendors. The calculation of annualized capital costs is helpful in other instances as well—​for example: • • •

when preparing to report unit costs for a given service when establishing fees intended to recover a specified portion of full costs for providing a given service when preparing a grant proposal or reporting annual costs for reimbursement purposes

Suggested for further information Columbia University, Public Technology Inc., and the International City Management Association. Evaluating Residential Refuse Collection Costs: A Workbook for Local Government. Washington, DC: Public Technology Inc., 1978. Hatry, Harry P., Donald M. Fisk, John R. Hall Jr., Philip S. Schaenman, and Louise Snyder. How Effective Are Your Community Services? Procedures for Performance Measurement, 3rd ed. Washington, DC: International City/​ County Management Association and The Urban Institute, 2006. See, especially, p. 179. Hatry, Harry P., Sumner N. Clarren, Therese van Houten, Jane P. Woodward, and Pasqual A. DonVito. Efficiency Measurement for Local Government Services. Washington, DC: Urban Institute, 1979. See “Cost Estimation Issues,” 175–​204. Kelley, Joseph T. Costing Government Services: A Guide for Decision Making. Washington, DC: Government Finance Officers Association, 1984.

a n n u a liz in g c o s t s of c a p i ta l i t e m s

Wang, Xiaohu. Financial Management in the Public Sector: Tools, Applications and Cases. Armonk, NY: M.E. Sharpe, 2006.

Web resources AssetWorks.com, “An Introduction to Useful Life and Depreciation” www. assetworks.com/​useful-​life-​and-​depreciation/​ Najjar, Dennis, “Depreciation: What Method to Choose and Is None an Option?”​ www.accountingdepartment.com/​blog/​depreciation-​what-​method-​to-​ choose Sage, “Government Accounting for Fixed Assets: GASB Guidelines for Your Organization” www.sage.com/​ n a/​ ~ /​ m edia/ ​ s ite/​Sage%20Fixed%20 Assets/​docs/​GASB_​Accounting.pdf Zarzycki, Nick, “How to Calculate Straight Line Depreciation (Formula)” https://​bench.co/​blog/​accounting/​straight-​line-​depreciation/​ Also, consider a web search using these key words: straight-​line depreciation, usage rate depreciation, annualizing capital costs. For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

235

19 IDENTIFYING FULL COSTS OF A PROGRAM

Local governments tend to understate the cost of individual programs or activities in much the same manner that most people would underestimate, say, the annual cost of operating their own automobiles.1 In these estimates, car owners would probably include the cost of gasoline and oil; they might also include their monthly payments or some appropriate component of the initial purchase price if they bought the car outright. But unless they are very careful, most people would slip up somewhere. They might forget to distribute the down payment over the life of the car or neglect to include interest charges.

1 Mohr and Rivenbark note that cost-​accounting weaknesses in local governments often render budget figures for individual departments “biased toward underestimation,” falling short of true cost. In the 1970s, E.S. Savas compared actual costs for various services with budget figures in 68 jurisdictions and discovered actual costs 30 percent greater than the figures shown on budget pages. See Zachary T. Mohr and William C. Rivenbark, “Contextualizing Cost Accounting in Government from a Historical Perspective,” in Cost Accounting in Government: Theory and Applications, ed. Zachary T. Mohr (New York: Routledge, 2017), 26–​45 and E.S. Savas, “How Much Do Government Services Really Cost?” Urban Affairs Quarterly 15, no. 1 (1979): 23–​42.

id enti f ying f ul l c o s t s o f a p ro g ra m

They might forget to include insurance premiums, the charge for the extended warranty, maintenance and repair charges, the cost of tires, the new paint job, the tag fee, the charge for an operator’s license, parking fees, or fines for occasional speeding tickets. Each of these costs is directly associated with the operation of a vehicle; omitting any would understate expenses. Other related expenses, such as the cost of a garage and driveway, in the case of homeowners, might not be eliminated by a decision to sell the car and use public transit; but in a full accounting of related expenses, these expenses should be included on a pro rata basis as well. Understating the cost of a program or activity in local government is rarely the product of deliberate deceit. More often it is simply the result of overlooking expenses that are related to a program but budgeted elsewhere or allocated in a previous fiscal year. Unless all expenses associated with a program are identified, the cost of that program will be understated and comparisons with similar programs offered by other entities, or even comparisons over time in a single jurisdiction, will be inaccurate and misleading. Incomplete estimates of program costs are especially troublesome when a local government’s decision makers are considering major program changes or service delivery alternatives and need a full-​cost bottom line.

Scenario: Barrow, Maine The city council of Barrow, Maine, had just adopted the budget for the upcoming year, and the budget director and her assistant were relaxing in her office, celebrating the high points and commiserating over the low points of the previous months. Suddenly, the budget director interrupted the conversation with what seemed a strange question. “What kind of a budget would you say we have?” “A program budget,” the budget analyst responded. “A program budget?” “Yes. Sure. We call them ‘activities,’ but each page describes a program and reports all of its costs.” “All of its costs?” “Yeah. I think so.” The budget director shook her head. “Three years ago, the city council adopted a policy calling for full-​cost recovery by the various internal

237

238

what does it cos t?

service funds and for all services having user fees. Departmental charges for equipment maintenance, information services, the print shop, and all the other internal services are supposed to cover all their costs. So are the user fees for fee-​supported outside services, such as the team fees for softball.” “We do that, don’t we?” “I don’t think so. We cover the amount shown on the program’s budget page, but I think we’re underreporting our costs. And, if I’m correct, then our fees are falling short of full-​cost recovery.” “What costs do you think we’re missing?” asked the budget analyst. “Well, I’m not sure we’re including all the costs of employee benefits with each program, for one thing. We’re not allocating costs for administrative overhead, for another. Some of the costs of the finance department, the human resources department, the city manager, and other administrators here at city hall should be allocated to the various programs that benefit from the services they provide. I’ll bet that we also badly understate various overhead costs associated with our facilities. We don’t even charge building rental.” “Building rental? The city owns these buildings. Why should a department have a rental charge?” “The building wasn’t free. Some form of rent or depreciation for all capital items would more accurately reflect the costs of providing a particular program.”

Determining full costs The budget director and budget analyst continued to discuss various costs that might be missed by relying on the current budget format for their service fee analyses. By the end of their conversation, the list was rather long, and the budget analyst had offered to explore the matter more systematically. A review of available reference materials led the budget analyst to the discovery of two alternative paths for assigning full costs: •

Activity-​based costing (ABC)—​ a rigorous costing method that typically relies on the logging of work activities, work sampling, or other means to achieve a high degree of precision in assigning direct and indirect costs to a work activity as part of a detailed accounting system.

id enti f ying f ul l c o s ts o f a p r o g ra m

•

Program costing through estimates and general allocation formulas (traditional cost accounting)—​a less rigorous method of assigning direct and indirect costs to activities or clusters of activities (that is, “programs”) based on allocation formulas deemed “reasonable.”

BOX 19.1  ACTIVITY-​BASED COSTING Developed in the private sector and designed especially to meet the needs of manufacturing industries, activity-​based costing (ABC) offers managers a higher degree of precision than traditional cost-​accounting systems in accounting for all the costs of a given product or service. This can be particularly valuable to private sector managers as they set out to emphasize profitable product lines, deemphasize less profitable lines, control overhead and production costs, make outsourcing decisions, and assign prices to their products. Some of these benefits of cost accounting apply to public sector managers, too. In a true ABC system, costs are assigned to a given activity (typically a distinct function within a program), based on that activity’s consumption of resources. All costs associated with the activity, whether direct (for example, materials and labor directly involved in production) or indirect (for example, administrative support within and beyond the department, as well as other overhead costs), are recorded either as they occur or by a method far more precise than general allocation formulas typical of traditional cost accounting systems. Some proponents of ABC draw a particularly sharp contrast with traditional cost accounting systems. Traditional cost accounting arbitrarily allocates nondirect overhead costs, a method that corrupts cost integrity. To ABC … allocation is a dirty word. ABC … resolves misallocations by using resource and activity drivers that reflect unique consumption patterns and link cause and effect to the cost-​assignment process.1 In earlier eras, overhead costs often were minor portions of total product costs. General approximations of overhead introduced only minor inaccuracies into the bottom line. In today’s economy, overhead commands a bigger share of the total cost of products and services. Apportioning overhead on a pro rata basis according to employee counts,

239

240

what does it cos t?

budget appropriations, or other arbitrary methods may introduce substantial inaccuracies. Labor-​intensive services might not require any greater overhead service than others, for example, but if overhead is assigned based on proportions of full-​time equivalent employees, full-​cost figures are inappropriately inflated. ABC offers an appealing level of precision that overcomes such inaccuracies, but the system imposes a heavy data collection burden that has limited its adoption in the public sector. Converting a traditional cost accounting system to ABC A city or county with a traditional cost accounting system might divide its recreation program to capture separate costs for its swimming pools, community center, and other recreation programs.2 Each would include direct costs for wages, supplies, and so forth, plus a share of the costs from the local government’s “overhead pools” (for example, buildings and maintenance, engineering, law, finance, and administration). Costs from the first two of these overhead pools—​buildings and maintenance, engineering—​might be divided among operating programs based on a given program’s share of total fixed assets; costs from the third and fourth overhead pools—​law, finance—​might be allocated based on a given program’s budget as a percentage of all nonoverhead budgets; and costs from the fifth pool—​administration—​assigned based on a given program’s share of all nonoverhead employees. Although each allocation formula in a traditional cost accounting system is tied to a generally plausible rationale, none reflects a cause-​ and-​effect relationship precisely. The time and effort of the director of administration and the costs incurred for printing, copying, and liability insurance—​all included in the overhead pool for administration—​are only roughly proportional to a program’s share of local government employees. Controversial programs often require disproportionate amounts of administrative time; programs with high risk exposure will require disproportionate liability insurance; and swimming pools may require fewer printing and copying services than other recreation programs. To convert this traditional cost accounting system to ABC, each of the three program categories would be subdivided to establish more precise activities within each program. Furthermore, each of the five overhead pools would be divided into activities so that overhead costs assigned to a program activity would be based not on an allocation formula but

id en ti f yin g f u l l c o s ts of a p ro g ra m

instead on the performance of an overhead activity that directly benefits the program. A word of caution Organizations often overstate their use of techniques deemed to be “progressive” or “advanced.” Over the years, for example, many have claimed the use of zero-​based budgeting, cost-​benefit analysis, and benchmarking on the strength of practices that only faintly resembled the actual technique. It is likely that some organizations adopting a formula-​based allocation approach to full-​cost accounting, commendable as that step may be, will nevertheless overstate their accomplishment by calling it ABC. Other local governments following their lead should understand that, although they may have substantially upgraded their cost accounting system and done so in a most pragmatic way, their revised system may still not constitute true activity-​based costing. 1 Gary Cokins, Alan Stratton, and Jack Helbling, An ABC Manager’s Primer: Straight Talk on Activity-​Based Costing (Montvale, NJ: Institute of Management Accountants, 1993), 11. 2 This example is based on an assessment of the advisability of upgrading the cost accounting system of the city of Wooster, Ohio, as reported in Richard E. Brown, Mark J. Myring, and Cadillac G. Gard, “Activity-​Based Costing in Government: Possibilities and Pitfalls,” Public Budgeting and Finance (Summer 1999): 3–​21.

The greater precision possible through ABC was appealing, but after learning a few details the analyst recognized that the data collection requirements of ABC would impose a heavy burden not only on the finance department but also on the operating departments. “Maybe someday we will be ready for activity-​based costing,” the analyst thought, “but right now, even the less ambitious approach will be a big step forward—​and will be a lot more practical to do.” Although he felt a little guilty for passing up the chance to promote something as “cutting edge” as ABC, the analyst decided it would be more prudent, given current staff resources, to propose a less ambitious step forward in cost accounting. He would recommend that the city more conscientiously track direct costs by program whenever possible and use allocation formulas for assigning less easily tracked direct costs as well as overhead costs to get a more complete picture of full costs (traditional cost

241

242

wh at doe s it cos t?

accounting). Accordingly, he set out to develop five worksheets for identifying the costs of a selected local government program or activity.

FROM THE ELECTRONIC TOOLKIT BOX 19.2  MAKING COST WORKSHEETS Some administrators will find that assembling cost worksheets can be simplified using techniques similar to those used for preparing work-​ distribution charts (see Box 11.1). To make a cost worksheet, enter the column labels and the data that correspond to the labels. (For directions on opening Excel and entering data refer to Box 1.1.) After the data are entered, Excel can do various calculations. In Worksheet 19.1 the figures in the right-​hand column indicate the products of the data in each row. For this worksheet the product for each row of data in columns B through D needs to be found and these products must appear in column E. In cell E2 enter: “=B2*C2*D2” in order to multiply the data from the previous columns. The product will appear in cell E2. Another method for multiplying data is to use the PRODUCT function. Simply enter the command and the range of data being multiplied. In this example, enter “=PRODUCT(B2:D2).” (For Microsoft Excel 2007 users, an alternative to typing this instruction would be to select the “PRODUCT” function from the “Math and Trig” category from the “Insert Function” option on the “Formulas” menu. For older versions of Excel, search for the “PRODUCT” function by selecting the “Function” option from the Insert menu.) To copy either formula for the other cells in the column, the calculation may be entered manually by retyping the commands and changing the names of the multipliers. For example, in cell E3 enter “=B3*C3*D3” or “=PRODUCT(B3:D3).” But there is a quicker method for entering these formulas. After entering the formula in cell E2, highlight that cell and move the cursor to the lower, right-​hand corner of the cell. Wait for a thin, black cross to appear. Then click and hold the mouse down and drag it down through the number of cells that need the same formula. Release the mouse and the products will be displayed.

id en ti f yin g f u l l c o s ts of a p ro g ra m

Other functions that may be useful in worksheets like these are SUM and QUOTIENT. These commands are used in exactly the same manner as described for the “product” function. Simply enter the command and then enter the range of the affected data. For example, Worksheet 19.1 calls for a grand total of the entries in the right-​hand column (the fifth column, alternatively called column E). To find the total of column E, enter “=SUM(E2:E13).” By using these functions in cost worksheets, calculations may be performed more easily and more reliably.

Worksheets Personal services—​salaries, wages, and fringe benefits—​are a major expense category in most local government programs. By compiling information that more accurately identifies employee time devoted to a given program and excludes time directed elsewhere, a better picture of program costs begin to emerge. Worksheet 19.1 calls for: 1. identification of all positions directly engaged in the selected program or activity 2. staffing of each such position in full-​time equivalents (FTEs) (for example, if the employee works 20 hours per week year round, the FTE is 0.5; if the position is filled by a seasonal employee working 40 hours per week for three months, the FTE is 0.25) 3. estimation of the percentage of each employee’s time on the job spent on the selected program (for example, a full-​time employee devoting 20 hours per week to the program would have 0.5 recorded in this column, as would a 20-​hour per week employee devoting 10 hours to the program) 4. notation of annual salary or wages for each position on a full-​time basis (or the average, if several employees engaged in the program have the same title) 5. computation of the product of columns 2, 3, and 4 (the sum of column 5 plus total overtime payments for the program captures the salaries and wages of personnel directly engaged in the program)

243

244

what does it cos t?

Worksheet 19.1  Salaries and Wages of Personnel for a Selected Program Activity Titles of positions directly engaged in selected program/​ activitya (1)

Number of employees (in full-​ time equivalents (2)

Fraction of time on the job spent on selected program/​ activity (3)

Annual expenditures per person-​year ($) (4)

Total ((2) x (3) x (4)) (5)

Total overtime payment for selected program activityb: Total salary and wage costs: Source: Adapted from Columbia University, Public Technology, Inc., and the International City Management Association, Evaluating Residential Refuse Collection Costs: A Workbook for Local Government and Worksheet Supplement (Washington, DC: Public Technology, Inc., 1978). a Include supervisory and support personnel (for example, secretaries, clerks, custodians) from the same department. b For overtime figure, see budget or financial records.

Worksheet 19.2 addresses fringe benefit costs. The person-​years devoted to the program and covered by specified benefits are multiplied by the cost per person-​year for those benefits to derive total fringe benefit costs. Worksheet 19.3 addresses other operating expenses. Common expenses such as supplies, building expenses, vehicles, equipment, insurance, and contractual services are identified on the form, but many programs have unusual expense items that should also be identified and included in the total. Capital items and internal services deserve special attention in calculating program expenses. The costs of acquiring and maintaining buildings and vehicles, for example, often are not reflected in the budget of the program

id enti f ying f ul l c o s t s o f a p ro g ra m

Worksheet 19.2  Fringe Benefit Costs for a Selected Program/​Activity Benefit (1)

FICA/​Social Security

Person years devoted to program/​activity that are covered by specified benefita (2)

Expenditures per person-​year ($) (3)

$

Total ((2) × (3)) (4)

$

Insurance A. Hospital B. Dental C. Life D. Disability E. Workers’ compensation F. Unemployment compensation Retirement contribution Supplemental retirement payments Deferred compensation /​401kb Uniforms and cleaning Safety equipment (for example gloves, shoes) Longevity or bonus pay Other post-​employment benefitsc Other Total fringe benefits $ Source: Adapted from Columbia University, Public Technology, Inc., and the International City Management Association, Evaluating Residential Refuse Collection Costs: A Workbook for Local Government and Worksheet Supplement (Washington, DC: Public Technology, Inc., 1978) and William C. Rivenbark, ed., A Guide to the North Carolina Local Government Performance Measurement Project (Chapel Hill: University of North Carolina Institute of Government, 2001). a Full-​time equivalent of employees who are engaged at least partially in the selected program and who are entitled to benefit coverage multiplied by the fraction of their time spent on that program. b Include here only if not charged to salaries/​wages (see Worksheet 19.1). c If a government chooses to fund in advance part or all of a plan to cover its other post-​ employment benefits (OPEB) on the basis of actuarial valuation, the annual allocation should be entered on this worksheet. An entry here represents a direct expenditure and should reflect the program’s proportional share of OPEB costs. Unless the government already accounts for this expenditure by individual programs or assesses a fee paid by the program to an internal service fund, a reasonable allocation formula must be devised. One approach for determining the program’s share of OPEB would be to assign costs proportionate to the program’s share of total current full-​time employees (the underlying assumption, then is that the program’s share of current retirees is roughly the same as its share of current employees). For more information on the handling of OPEBs, see Gregory Allison, “To Tell the Truth: How Much in Benefits Are Governments Promising Retirees?” Popular Government 73, no. 1 (Fall 2007): 14–​17.

245

246

what does it cos t?

Worksheet 19.3  Other Operating Expenses for a Selected Program/​Activity Category (1) Supplies _​_​_​_​_​_​_​_​_​_​_​_​_​_​ other Building expenses telephone utilities renta _​_​_​_​_​_​_​_​_​_​_​_​_​_​ Other Vehicles renta operationb maintenanceb Other equipment maintenance and repair _​_​_​_​_​_​_​_​_​_​_​_​_​ Purchases for resale Training and travel Fees and licenses Advertising Uniform purchase or rental Dues/​membership/​subscriptions MIS/​data processing/​IT/​GIS

 

Insurance _​_​_​_​_​_​_​_​_​_​_​_​_​_​

 

Contractual services _​_​_​_​_​_​_​_​_​_​_​_​_​_​

Total annual expenses (2)

% Applicable to selected program (3)

Total (2) x (3) (4)

id enti f ying f ul l c o s t s o f a p ro g ra m

Worksheet 19.3 Cont. Category (1)

Total annual expenses (2)

% Applicable to selected program (3)

Total (2) x (3) (4)

Contract administration Other operating expenses  

_​_​_​_​_​_​_​_​_​_​_​_​_​_​  

_​_​_​_​_​_​_​_​_​_​_​_​_​_​ Totalc $ Source: Adapted from Columbia University, Public Technology, Inc., and the International City Management Association, Evaluating Residential Refuse Collection Costs: A Workbook for Local Government and Worksheet Supplement (Washington, DC: Public Technology, Inc., 1978) and William C. Rivenbark, ed., Guide to the North Carolina Local Government Performance Measurement Project (Chapel Hill: University of North Carolina Institute of Government, 2001). a If the local government is renting office space and vehicles for this program, those amounts should be entered here. If the local government possesses its own program facilities and vehicles, it is still appropriate to enter “rental” of some kind (perhaps based on annual allocations for replacement, on a depreciation schedule, on debt service, or on market rates) rather than to imply that they are “free.” b Fuel, parts, and maintenance expenses should be recorded here, except when such expenses would duplicate previous entries (for example, if repair personnel are listed on Worksheet 19.1, labor charges should not be repeated here). c “Cost recovery” is excluded from the tabulation of operating expenses in this worksheet. Fairness of comparison among multiple programs or options will demand uniform treatment of program receipts, sometimes labeled “cost recovery” and reported in the budget as an expenditure reduction and sometimes labeled “general revenue” and having no such effect on reported program expenditures. Uniformity is achieved here by treating all receipts as revenues to be considered elsewhere, rather than as a reduction of operating expenses.

that benefits from their use. The building may have been constructed years ago with resources from the sale of bonds, the debt service for which is budgeted in another activity. Automobiles assigned to the program may have been purchased last year, with this year’s budget showing no acquisition costs whatsoever. The expense of maintaining buildings and vehicles may be reflected only in the budgets of the internal service departments responsible for those maintenance activities. In such cases, program expenses are understated, making the program appear less costly than it actually is. A more accurate statement of program expenses would assign proportionate capital and maintenance expenses to the programs that derive benefits from those facilities and equipment. Various options exist for

247

248

what does it cos t?

Table 19.1  Rule-​of-​Thumb Alternatives for Annualizing Capital Costs Costs to be annualized

Rule of thumb options

Furniture and office equipment Maintenance/​construction equipment Automobiles and light equipment Medium/​heavy motor equipment Data processing equipment Light/​miscellaneous equipment Other equipment Rental equipment Buildings

10 percent of acquisition cost 12 percent of acquisition cost 30 percent of acquisition cost 16 percent of acquisition cost 20 percent of acquisition cost 10 percent of acquisition cost based on judgment of useful life   report rental payments 2 percent of the original construction cost plus capitalized renovations report rental payments

Rental space

Source: William C. Rivenbark and K. Lee Carter, “Benchmarking and Cost Accounting: The North Carolina Approach,” Journal of Public Budgeting, Accounting and Financial Management 12, no. 1 (2000): 132. Reprinted with permission. Note: These rules of thumb are used to simplify and standardize cost accounting procedures among the cities and counties participating in the North Carolina Local Government Performance Measurement Project. A local government attempting to identify the full costs of an activity should also consider annual debt service for capital items secured through debt financing. Alternatively, annual capital costs for an activity could be derived more precisely using depreciation calculations described in Chapter 18.

determining reasonable assessments. Annual costs for building “rental,” for example, may be the program’s proportionate share (based on square footage) of an annualized capital assessment tied to the building’s acquisition or replacement cost. Alternatively, they may be based on the annual debt service requirements—​principal and interest—​to retire the bonds sold to finance construction of the building. Vehicle “rental” may be based on a depreciation schedule (see Chapter 18) or annual contributions to a vehicle replacement fund in amounts sufficient to have enough money available to purchase a replacement vehicle when the current vehicle is no longer serviceable. An alternative approach for annualizing selected capital costs, based on simple rules of thumb, is shown in Table 19.1. Vehicle maintenance assessments should be based on actual costs or a proportionate share of general maintenance expenses. Worksheet 19.4 addresses the tabulation of overhead costs—​a program’s proportionate share of the expense of various administrative and support services provided by other departments of the local government. The city manager’s time, for example, is divided among the various programs of the local government. Similarly, the human resources department provides

id enti f ying f ul l c o s t s o f a p ro g ra m

Worksheet 19.4  Overhead Costs for Selected Program/​Activity Overhead agency

Total expenditures of overhead agency

Basis of allocation to selected program/​activitya

Overhead agency expenditures allocated to selected program/​ activity

Executive/​mayor/​ manager

$_​_​_​_​_​​_​_​_​_​_​

Program’s FTEs as percentage of total FTEsb

$_​_​_​_​_​_​_​_​_​_​

Governing body/​ council

_​_​_​_​_​_​_​_​_​_​_​_​ Program’s FTEs as percentage of total FTEsb

_​_​_​_​_​_​_​_​_​_​_​

Finance, comptroller, _​_​_​_​_​_​_​_​_​_​_​_​ Program’s FTEs as percentage budget, treasurer of total FTEsb

_​_​_​_​_​_​_​_​_​_​_​

Billing and collections

_​_​_​_​_​_​_​_​_​_​_​_​ Percentage of expenditures attributable to program/​ activity

_​_​_​_​_​_​_​_​_​_​_​

Clerk

_​_​_​_​_​_​_​_​_​_​_​_​ Program’s FTEs as percentage of total FTEsb

_​_​_​_​_​_​_​_​_​_​_​

Attorney/​legal

_​_​_​_​_​_​_​_​_​_​_​_​ Program’s FTEs as percentage of total FTEsb

_​_​_​_​_​_​_​_​_​_​_​

Central support _​_​_​_​_​_​_​_​_​_​_​_​ Percentage of stock services (for requisitions, postage example, charges, copies attributable stockroom, mail to program activity services, printing)

_​_​_​_​_​_​_​_​_​_​_​

General services

_​_​_​_​_​_​_​_​_​_​_​_​ Program’s FTEs as percentage of total FTEsb

_​_​_​_​_​_​_​_​_​_​_​

MIS/​data processing/​IT/​ GISc

_​_​_​_​_​_​_​_​_​_​_​_​ Program’s FTEs as percentage of total FTEsb,c

_​_​_​_​_​_​_​_​_​_​_​

Purchasing

_​_​_​_​_​_​_​_​_​_​_​_​ Program’s number of purchase orders as percentage of total

_​_​_​_​_​_​_​_​_​_​_​

Human resources/​ personnel

_​_​_​_​_​_​_​_​_​_​_​_​ Program’s FTEs as percentage of total FTEsb

_​_​_​_​_​_​_​_​_​_​_​

Risk management

_​_​_​_​_​_​_​_​_​_​_​_​ Program’s FTEs as percentage of total FTEsb

_​_​_​_​_​_​_​_​_​_​_​

Liability insurance

_​_​_​_​_​_​_​_​_​_​_​_​ Program’s FTEs as percentage of total FTEsb

_​_​_​_​_​_​_​_​_​_​_​ (continued)

249

250

what does it cos t?

Worksheet 19.4 Cont Overhead agency

Total expenditures of overhead agency

Basis of allocation to selected program/​activitya

Property insurance

_​_​_​_​_​_​_​_​_​_​_​_​ Program’s square footage as percentage of total

Insurance on equipment and vehicles

_​_​_​_​_​_​_​_​_​_​_​_​ Program’s vehicles/​equipment as percentage of total

Other post-​ employment benefitsd

_​_​_​_​_​_​_​_​_​_​_​_​ Program’s FTEs as percentage of total FTEs

Overhead agency expenditures allocated to selected program/​ activity

Total overhead costs: $_​_​_​_​_​_​_​_​_​_​​ Source: Adapted from Columbia University, Public Technology, Inc., and the International City Management Association, Evaluating Residential Refuse Collection Costs: A Workbook for Local Government and Worksheet Supplement (Washington, DC: Public Technology, Inc., 1978) and William C. Rivenbark, ed., A Guide to the North Carolina Local Government Performance Measurement Project (Chapel Hill: University of North Carolina Institute of Government, 2001). Note: FTE = full-​time equivalent employee. MIS = management information services. IT = information technology. GIS = geographic information system. a An alternate basis of allocation may be chosen that provides a more reasonable representation of actual expenditures attributable to the support of the selected program. See Table 19.2. b Alternatively, the selected program/​activity’s salaries and wages as a percentage of the local government’s salary and wage total. c Include here only if not included as a direct charge on Worksheet 19.3. d If a government chooses to fund in advance part or all of a plan to cover its other post-​ employment benefits (OPEB) on the basis of actuarial valuation, the annual allocation should be entered on this worksheet. An entry here represents a direct expenditure and should reflect the program’s proportional share of OPEB costs. Unless the government already accounts for this expenditure by individual programs or assesses a fee paid by the program to an internal service fund, a reasonable allocation formula must be devised. One approach for determining the program’s share of OPEB would be to assign costs proportionate to the program’s share of total current full-​time employees (the underlying assumption, then, is that the program’s share of current retirees is roughly the same as its share of current employees). For more information on the handling of OPEBs, see Gregory Allison, “To Tell the Truth: How Much in Benefits Are Governments Promising Retirees?” Popular Government 73, no. 1 (Fall 2007): 14–​17.

id enti f ying f ul l c o s t s o f a p ro g ra m

services on an organization-​wide basis. Each program derives benefits from these services and is spared the trouble and expense of performing these functions itself. It is only proper that a proportionate share of the costs of overhead agencies be borne by the various programs of the local government on some proportionate basis. The basis used in Worksheet 19.4 varies from one overhead item to another, although most frequently it is the program’s number of full-​time equivalent employees as a percentage of the entire local government workforce. The assumption—​an imperfect one, at best—​is that a program that includes 10 percent of the local government’s workforce probably also receives about 10 percent of the benefit of the city manager’s office, the finance department, the city clerk’s office, and other overhead agencies. Other bases of allocation are used for some overhead items on Worksheet 19.4, with still other options noted in Table 19.2. Worksheet 19.5 combines the figures from the other four worksheets to arrive at a cost summary for the program. That total may differ substantially from the amount listed for the activity in the local government’s annual budget. It provides a much improved basis for calculating efficiency measures (for example, unit costs), for comparing program costs with those of other jurisdictions (assuming they use similar cost-​accounting methods), for documenting program costs in grant applications, and for developing service fees.

Worksheet 19.5  Cost Summary for Selected Program/​Activity Cost source

Dollars

Total personnel salaries and wages from Worksheet 19.1 Total fringe benefits from Worksheet 19.2 Other operating costs from Worksheet 19.3 Total overhead costs from Worksheet 19.4 Total cost Source: Adapted from Columbia University, Public Technology, Inc., and the International City Management Association, Evaluating Residential Refuse Collection Costs: A Workbook for Local Government and Worksheet Supplement (Washington, DC: Public Technology, Inc., 1978).

251

252

what does it cos t?

Table 19.2  Possible Allocation Bases for Various Indirect (Overhead) Cost Assignments Costs

Allocation base

Accounting

Number of invoices attributable to a given activity, as a percentage of all invoices Fixed assets of the activity, as a percentage of all fixed assets Square feet of space assigned to the activity, as a percentage of total square feet maintained by custodial service Percentage of central processing unit time devoted to the activity Fixed assets of the activity, as a percentage of all fixed assets Budgeted expenses of the activity, as a percentage of the local government’s entire budget, or number of accounting transactions attributable to the activity, as a percentage of all transactions Number of full-​time equivalent (FTE) employees assigned to the activity, as a percentage of the entire workforce Number of FTEs assigned to the HR activity, as a percentage of the entire workforce Number of FTEs assigned to the activity, as a percentage of the entire workforce Budgeted expenses of the activity, as a percentage of the local government’s entire budget Number of FTEs assigned to the activity, as a percentage of the entire workforce Number of FTEs assigned to the activity, as a percentage of the entire workforce Number of purchase orders attributable to the activity, as a percentage of all purchase orders Square feet of space assigned to the activity, as a percentage of total square feet Number of phones assigned to the activity, as a percentage of all telephones Number of checks attributable to the activity, as a percentage of all checks Square feet of space assigned to the activity, as a percentage of total square feet Number of miles driven on tasks associated with the activity, as a percentage of all miles drivena

Building maintenance Custodial service Data processing Engineering Finance

General administration Human resources Insurance Law department Mayor/​Manager’s office Payroll preparation Purchasing Rent Telephone Treasurer Utilities Vehicles

Sources: Adapted from Mark D. Abrahams and Mary Noss Reavely, “Activity-​Based Costing: Illustrations from the State of Iowa,” Government Finance Review 14, no. 2 (1998): 16; Richard E. Brown, Mark J. Myring, and Cadillac G. Gard, “Activity-​Based Costing in Government: Possibilities and Pitfalls,” Public Budgeting and Finance 19 (Summer 1999): 3–​21; and William C. Rivenbark and K. Lee Carter, “Benchmarking and Cost Accounting: The North Carolina Approach,” Journal of Public Budgeting, Accounting and Financial Management 12 (Spring 2000): 130. a Alternatively, vehicle costs for the activity could be derived from actual maintenance and repair costs incurred for the upkeep of the activity’s vehicles, plus annual capital costs based on depreciation (see Chapter 18).

id enti f ying f ul l c o s ts o f a p r o g ra m

Utility of full-​c ost identification A true program budget reports all the costs associated with a given program. A basic premise of such a system is that full-​cost reports will help policy makers in their deliberations regarding needs, priorities, and options. Systems that presume to report full costs but fail to do so are deceptive and may contribute to ill-​advised decisions. Unless they have identified the full costs of a program, officials are unable to answer some very basic questions: How much are we paying for this service? What is the cost per unit of service? How does our cost compare with that of other jurisdictions? How does the cost of this service compare with the costs of various other services offered by our city or county government? How much of the cost is being recovered by current service fees?

BOX 19.3  UNIFORM COST ACCOUNTING FOR COMPARATIVE PERFORMANCE MEASUREMENT PROJECTS Several projects undertaken in recent decades have been carefully designed to permit reliable comparison of cost and performance data among participating local governments. Examples include projects in North Carolina and Florida. To achieve their objective of cost comparability, each project had to secure agreement among participants on a set of uniform cost accounting rules. Otherwise, cost elements assigned to a given program by one jurisdiction might be counted differently or even excluded from the tabulation of program costs reported by a counterpart, thereby distorting the comparison. Individual local governments not part of a cooperative project that attempt to compare their own performance and cost data with data collected independently by other governments face a formidable task. Measures of service quality often can be compared with relatively few compatibility problems. Similarly, measures of efficiency that relate units of service to units of employee time (for example, staff-​hours per application processed, curb-​miles swept per operator-​hour) are easily compared in most cases. In contrast, cost data or measures of efficiency that rely on cost accounting systems (for example, unit costs for a given service)

253

254

what does it cos t?

are less easily compared because of wide variation in cost accounting practices from one local government to another. The desire to resolve this incompatibility is a major reason for establishing a cooperative performance measurement project.

BOX 19.4  OTHER APPLICATIONS OF COST ACCOUNTING Cost accounting systems that identify the full costs associated with a given service or activity facilitate analysis and bolster accountability. Among the uses of such systems are: • • • •

efficiency measurement, when combined with output data, for performance reports and operations reviews analysis of expenditure trends for a given service, including changes in unit costs over time the establishment of user fees to recover all or a specified portion of service expenses reconsideration of resource allocations in light of citizen priorities, service demands, and more precise cost information

References Abrahams, Mark D., and Mary Noss Reavely. “Activity-​Based Costing: Illustrations from the State of Iowa.” Government Finance Review 14, no. 2 (1998): 15–​20. Allison, Gregory S. “To Tell the Truth: How Much in Benefits Are Governments Promising Retirees?” Popular Government 73, no. 1 (2007): 14–​17. Brown, Richard E., Mark J. Myring, and Cadillac G. Gard. “Activity-​Based Costing in Government: Possibilities and Pitfalls.” Public Budgeting and Finance 19, no. 2 (1999): 3–​21. Cokins, Gary, Alan Stratton, and Jack Helbling. An ABC Manager’s Primer: Straight Talk on Activity-​Based Costing. Montvale, NJ: Institute of Management Accountants, 1993. Columbia University, Public Technology, Inc., and the International City Management Association. Evaluating Residential Refuse Collection Costs: A Workbook for Local Government and Worksheet Supplement. Washington, DC: Public Technology, 1978.

id enti f ying f ul l c o s t s o f a p ro g ra m

Mohr, Zachary T., and William C. Rivenbark. “Contextualizing Cost Accounting in Government from a Historical Perspective.” In Cost Accounting in Government: Theory and Applications, edited by Zachary T. Mohr, 26–​45. New York: Routledge, 2017. Rivenbark, William C., ed. A Guide to the North Carolina Local Government Performance Measurement Project. Chapel Hill: University of North Carolina Institute of Government, 2001. Rivenbark, William C., and K. Lee Carter. “Benchmarking and Cost Accounting: The North Carolina Approach.” Journal of Public Budgeting, Accounting and Financial Management 12, no. 1 (2000): 125–​137. Savas, E.S. “How Much Do Government Services Really Cost?” Urban Affairs Quarterly 15, no. 1 (1979): 23–​42.

Suggested for further information Bandy, Gary. Financial Management and Accounting in the Public Sector, 2nd ed. New York: Routledge, 2015. See Chapter 5, “Costing and Project Appraisal,” pp. 177–​203. Lee, Bethany K. “Introduction to Activity-​Based Costing,” IQ Report. Washington, DC: ICMA, February 1998. Michel, R. Gregory. Cost Analysis and Activity-​Based Costing for Government. Chicago: Government Finance Officers Association, 2004. See, especially, Chapter 7, “Setting Fees and Charges,” and Chapter 12, “Activity-​Based Costing.” Mohr, Zachary T. (ed.). Cost Accounting in Government: Theory and Applications. New York: Routledge, 2017. USEPA. Full Cost Accounting for Municipal Solid Waste Management: A Handbook. Report no. 530-​R-​95-​041. Washington, DC: US Environmental Protection Agency, September 1997. Wang, Xiaohu. Financial Management in the Public Sector: Tools, Applications and Cases. Armonk, NY: M.E. Sharpe, 2006. Webster, Douglas W., and Gary Cokins. Value-​Based Management in Government. Hoboken, NJ: John Wiley & Sons, 2020.

255

256

what does it cos t?

Web resources Briner, Russell F., Mark Alford, and Jo Anne Noble, “Activity-​Based Costing for State and Local Governments” ​www.imanet.org/​-​/​media/​762fa66b75174f 69a7d8734418fbef38.ashx Kaplan, Robert S., and Steven R. Anderson, “Time-​Driven Activity-​Based Costing” https://​hbr.org/​2004/​11/​time-​driven-​activity-​based-​costing Marlowe, Justin, “The ABCs of Cost Accountability” ​www.governing.com/​ columns/​public-​money/​gov-​cost-​accountability-​alphabet.html Also, consider a web search using these key words: activity based costing and full cost accounting. For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

20 CALCULATING GO-​AWAY COSTS FOR PRIVATIZATION DECISIONS

Calculating the full costs of a program, including indirect and capital costs (see Chapter 19), strengthens the ability of local government officials to gauge program efficiency and assess the adequacy of fees for services. However, using the full costs of an in-​house operation as the basis for a decision to contract for that service from an outside entity could lead the local government to a major mistake. Several local governments at the forefront of efforts to apply the privatization tool wisely—​including the cities of Phoenix, Indianapolis, and Charlotte—​ refrain from using full costs in such deliberations, opting instead for what they call go-​away costs as the basis for their decisions. They recognize that in preparing for a privatization decision, two categories of local government costs must be identified: (1) contract administration costs, which include the costs of administering the contract and monitoring performance; and (2) go-​away costs, which include only that portion of full costs of an in-​house program or service that will actually go away when the service is produced by an outside entity. The sum of contract administration costs and the bid price of an outside entity may be compared with go-​away costs to assess the net gain or loss from a decision to contract. This procedure excludes from consideration all costs that will continue to be incurred by the local government following initiation

258

what does it cos t?

of the contract. For example, a full-​cost calculation for a given program probably includes portions of the salaries of the city or county manager, finance director, human resources director, and other management officials. It also includes annualized costs for capital equipment and facilities. In the event that the local government decides to contract for a given service, it is doubtful that the salaries of any of the key officials listed above will be reduced. Their costs will be redistributed among continuing programs, but they will not go away. But if the contracted program is large, it is possible that the staff of payroll clerks in the finance department or personnel specialists in the human resources department might be reduced as a result. If previously shared capital equipment will now be retained by the other department and if vacated space will be filled by other in-​house programs without any net reduction in local government expenditures, then these costs, although appropriately included in an accounting of full costs, should be excluded from go-​away costs.

Scenario: Buckaroo, Wyoming “How could this happen?” wondered Herb Jada, budget officer for the city of Buckaroo. “This is going to be embarrassing.” Herb was already dreading the thought of trying to explain the budgeting fiasco that followed Buckaroo’s heralded foray into privatization. Last fall the city had solicited bids from vendors for residential refuse collection services, fleet maintenance, and custodial services. The city had tried to be thorough in its analysis of the contracting option and, at the urging of vendors, had carefully calculated the full costs of the in-​ house refuse collection and fleet maintenance programs, including costs of facilities, capital equipment, and overhead. After all, the vendors had reminded the city that contractor costs for these items would be included in their bids, so full cost accounting by the city would simply ensure “a level playing field.” When bids were received and the cost comparison showed an advantage for privatization (see  Table 20.1), Herb and others had eagerly recommended entering all three contracts. But now, putting the numbers for next year’s budget together six months later, something is wrong. In frustration, Herb called a former classmate who was now on the budget staff of a city that had engaged in privatization for several years. “I’m at a loss, Bob, and, unfortunately, so is our budget,” Herb began. “We were planning our budget around an anticipated $380,000 in increased

g o- aw ay cos ts f o r p ri vat i z at i o n d e c i s i o n s

Table 20.1  Full Cost of In-​House Operation Compared to Low Bids In-​house full costs ($) Custodial services Salaries/​wages Fringe benefits Other operating costs Overhead Low bid Contract administration Total Fleet maintenance Salaries/​wages Fringe benefits Other operating costs Overhead Low bid Contract administration Total Residential refuse collection Salaries/​wages Fringe benefits Other operating costs Overhead Low bid Contract administration Total

Contract costs ($)

Difference

142,000 5,000 $147,000

$11,591

156,500 11,000 $167,500

$68,782

90,854 20,798 24,125 22,814

$158,591 136,914 30,216 34,031 35,121

$236,282 230,851 56,851 901,852 203,543

$1,393,097

1,244,000 14,000 $1,258,000 TOTAL

$135,097 $215,470

expenditures in the upcoming fiscal year, but that was supposed to be partially offset by $215,000 in savings from decisions to contract out three formerly in-​house services. I was anticipating the need for about $165,000 in new revenues, but the numbers come out to be more like $310,000 in needed revenue expansion. I cannot figure out where $145,000 of our contract savings went!” After asking a few questions, Herb’s friend spotted the problem. “You used full costs in your analysis rather than go-​away costs,” Bob said. “Look at the things you included as overhead costs—​shares of the expenses of the manager’s office, the finance department, things like that. Were the expenditures of any of those departments reduced following initiation of the contracts?”

259

260

what does it cos t?

“No, they weren’t,” Herb replied. “I see what you mean. We included them in the figures for in-​house services in order to show our full costs, but most of those costs stayed in the organization even when the decision was made to contract out these three services. No one asked the city manager and finance director to take a cut in pay just because a portion of their salaries had been in the overhead for a service that is no longer performed by a city department.” “Did the fleet maintenance contractor move into the city garage?” Bob inquired. “No, they have their own facility.” “If you included facility costs in your full costing, did they go away with the decision to contract?” “No, we still have the garage. The public works department is moving some of its offices and equipment in there. Those costs aren’t going away either.” “You might also check to see if some transitional costs are affecting your budget,” Bob suggested. “It’s usually a good idea to handle staff reductions associated with privatization through attrition and reassignment, if you can, but sometimes that means carrying some extra personnel costs for a while. You might find some of the expenses there.” “It’s a mess, isn’t it, Bob?” “Maybe, but you still might find that contracting for these services was a good idea—​just not as beneficial as originally thought. This won’t be a message that will be fun to deliver, but it is better that you discovered it rather than an elected official or a local critic.” “Thanks, Bob.”

Reanalysis in Buckaroo Herb reanalyzed the contracting decisions, this time using go-​away costs rather than full costs (Table 20.2).1 The new numbers showed that Bob’s prediction had been two-​thirds correct: contracting out the services had been 1 Distinguishing costs that will go away from those that will stay is less a matter of accounting than a process involving analysis of operations and negotiation with departmental officials who may try to preserve resources by keeping them out of the go-​away category.

g o- aw ay cos ts f o r p ri vat i z at i o n d e c i s i o n s

Table 20.2  Go-​Away Costs Compared to Contract Costs In-​house Full costs ($) Custodial services Salaries/​wages Fringe benefits Other operating costs Overheadb Low bid Contract administrationc Total Fleet maintenance Salaries/​wages Fringe benefits Other operating costs Overheadb Low bid Contract administrationc Total Residential refuse collection Salaries/​wages Fringe benefits Other operating costs Overheadb Low bid Contract administrationc Total

90,854 20,798 24,125 22,814

Go-​away costs ($)

Savings via contractinga ($)

90,854 20,798 24,125 4,200

$158,591

$139,977

136,914 30,216 34,031 35,121

136,914 30,216 34,031 16,542

$236,282

$217,703

230,851 56,851 901,852 203,543

230,851 56,851 901,852 95,054

$1,393,097

Contract costs ($)

$1,284,608

142,000 5,000 $147,000

−$7,023

156,500 11,000 $167,500

$50,203

1,244,000 14,000 $1,258,000 TOTAL

$26,608 $69,788

a Savings are the difference between go-​away costs and contract costs. b Not all overhead costs would disappear due to retained capital costs for space and some equipment. c The contract administration costs estimated by Buckaroo officials are minimal. Some authorities suggest 10 percent of the contract amount as a rule of thumb.

the proper choice in two of the three services. Herb was now convinced, however, that Buckaroo should have retained custodial services in-​house. Luckily, that was the smallest of the three contracted operations. The new numbers also solved the mystery of the missing $145,000 in anticipated savings. Actual savings would be about $70,000 rather than $215,000. Gathering his tables and his courage, Herb began walking down the hall to tell the city manager about his discovery. “This has been a valuable lesson,” he thought. “I just wish I had learned it some easier way.”

261

262

what does it cos t?

BOX 20.1  OTHER APPLICATIONS OF GO-​AWAY COSTS ANALYSIS The usefulness of go-​away costs is limited only by the range of possible contracting options in local government, which is to say the potential use of this technique is broad indeed. Although vendors will argue for full-​cost comparisons (see Chapter 19), prudent cities and counties will make all their contracting decisions based on go-​away costs instead.

Suggested for further information Martin, Lawrence L. “Evaluating Service Contracting.” In Management Information Service Report 25, no. 3, 1–​14. Washington, DC: International City/​County Management Association, March 1993. Michel, R. Gregory. Cost Analysis and Activity-​Based Costing for Government. Chicago: Government Finance Officers Association, 2004. See Chapter 8, “Make-​versus-​Buy Decisions,” 107–​117. Mohr, Zachary T. (ed.). Cost Accounting in Government: Theory and Applications. New York: Routledge, 2017.

Web resources Martin, Lawrence, “How to Compare Costs Between In-​House and Contracted Services” https://​reason.org/​wp-​content/​uploads/​files/​ba9b1490316698 83385db8c2ff63b9ef.pdf Rebus Community, “Cost Analysis” https://​press.rebus.community/​ financialstrategy/​chapter/​cost-​analysis/​ Also, consider a web search using these key words: go away costs, full costs, marginal costs. For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

21 LIFE-​CYCLE COSTING

Local governments are consumers and, like individual consumers, frequently purchase the cheapest of several similar items performing the same function, believing that the lowest-​priced item is the best buy. Sadly, however, this is not always true because the cost of owning an item includes more than its purchase price. Life-​cycle costing is a technique used by many businesses and governments to determine the total cost of owning an item, including costs associated with the item’s acquisition, operation, and maintenance. The technique not only accounts for the purchase price of the item but also identifies hidden costs of ownership. This chapter describes the life-​cycle approach to costing, suggests when the use of this technique is most appropriate, and offers a simple formula for calculating the lifetime cost of owning an item.

264

what does it cos t?

When to apply the technique of life-​c ycle costing Life-​cycle costing can be applied to many local government purchases but is most frequently used to determine the lifetime costs of moderately expensive, energy-​consuming equipment, including motor vehicles, climate-​control systems, data processing equipment, and lighting systems. Table 21.1 illustrates how considering energy costs can affect a potential purchaser’s assessment of two 15-​horsepower electric motors. Based solely on the purchase price, the motor offered by vendor A seems less expensive than the motor offered by vendor B. However, motor A has a simpler two-​pole system with a higher rate of energy consumption (12.5 kilowatts per hour) than the more efficient six-​pole motor B (12.1 kilowatts per hour)—​a very important factor in this decision because the purchasing local government plans to run the motor 10 hours a day, five days a week (that is, 2,600 hours per year). As shown in Table 21.1, motor A will actually cost $1,380 more than motor B over their lifetimes, assuming equal maintenance costs. Even a small efficiency difference can have an outsized effect when the energy costs of operation are far larger than the purchase price. The underlying concepts of life-​ cycle costing may also be used to strengthen performance standards in bid specifications. The specifications should call for the bidders to provide all the technical information necessary to project life-​cycle costs. While it may be helpful to give a reference brand Table 21.1  Supplementing Purchase Price with Lifetime Energy Costs Life-​cycle cost

Motor from Vendor A

Motor from Vendor B

Horsepower RPM Bid price Duty cycle

15 3,600 $996 2,600 hours per year 15 years 89.5% 12.5 $68,250

15 1,200 $1,800 2,600 hours per year 15 years 92.5% 12.1 $66,066

$69,246

$67,866

Life Efficiency rating Energy consumption (kilowatts per hour) Energy costs (kilowatt hour consumption rate × $0.14/​kilowatt hour × 39,000 hours) Life-​cycle cost (bid cost + energy cost) Life-​cycle cost difference ($69,246 –​67,866) = $1,380

l i f e -c y c l e c o s t i n g

name to act as a guideline to potential bidders, care should be taken to avoid preempting the competitive process. In the bid invitation, the manager or local government purchasing agent should require each potential supplier to submit documentation regarding an item’s expected energy consumption rate and the anticipated useful life of the item, assuming a given duty cycle (how much use it is expected to get over a one-​year period).

Preparing a life-​c ycle cost analysis Three cost factors—​acquisition cost, lifetime maintenance costs, and energy costs—​and salvage value are core elements of a life-​cycle cost analysis. The acquisition cost of an item includes its purchase price and transportation and installation costs. The acquisition cost should also reflect discounts in the purchase price and credit for trade-​in equipment. An item’s projected lifetime maintenance and energy costs are the anticipated costs for keeping the item in operable condition and for energy consumed in operating the item. Salvage value is also fundamental to the analysis. How much can the local government recoup by selling the item at the end of its projected life? Adding the three cost factors and subtracting the salvage value provides a simplified version of the life-​cycle cost of an item (see Table 21.2). Calculating life-​cycle costs based solely on acquisition costs, maintenance and energy costs, and salvage value may be sufficient for most analyses. For especially large or otherwise significant purchases, however, managers may find it useful to examine, when applicable, seven other costs associated with ownership: • • • • •

•

failure costs, including downtime, production losses, and rental costs training costs for personnel training in equipment usage, including tuition, time away from the job, meals, transportation, and lodging consumable supply costs arising from an item’s use storage costs for the item or for repair parts secondary costs for disposal of by-​products associated with the item’s use (such costs may be positive or negative—​for example, waste heat can be used to reduce energy consumption in colder months, reducing overall costs) labor costs or the wages and benefits for employees engaged in the operation of an item

265

266

what does it cos t?

Table 21.2  Formula for Life-​Cycle Costing The basic life-​cycle cost formula is Life-​cycle costs

= acquisition cost + lifetime maintenance costs + lifetime energy costs –​salvage value

where   Acquisition costs

= purchase price + transportation cost + installation cost –​ trade-​ins and discounts   Lifetime maintenance costs = anticipated costs of keeping the item in operable condition   Lifetime energy costs = energy consumption rate x cost of energy × duty cycle × life of the item   Salvage value = anticipated worth at the end of the item’s projected life The components of the lifetime energy costs are   Energy consumption rate   Cost of energy   Duty cycle  Life

= the rate at which energy is consumed (kilowatts per hour) = dollars per energy unit (cents per kilowatt hour) = annual number of hours item is used (number of hours in use per day × number of days in use) = length of time until item is replaced (number of years in use based on the duty cycle)

Source: Adapted from League of California Cities, A Guide to Life Cycle Costing: A Purchasing Technique That Saves Money (Sacramento: League of California Cities, December 1983), 3–​4. Adapted by permission of the League of California Cities.

•

money costs, including interest paid for a loan to purchase the item or interest forgone on money that could be invested elsewhere if not tied up in equipment purchases

The following example demonstrates the use of a life-​cycle costing formula that considers acquisition cost, maintenance and energy consumption costs, salvage value, and one other cost factor—​failure cost.

Scenario: Lindsey, Virginia The central water heater in the Municipal Utilities Building of Lindsey, Virginia, is on its last legs. “We’re lucky that unit didn’t give out last year,” Utilities Director Vance Carter remarked, as he invited his administrative analyst Antawn Jamestown to have a seat. “The purchasing agent got some bids for replacing the water

l i f e -c y c l e c o s t i n g

heater, and I would like for you to check out our options. We promote conservation, efficiency, and green options with our customers all the time, so I encouraged purchasing to seek energy-​efficient bids and help us practice what we preach.” He handed Antawn the bid file and said, “Do some life-​cycle costing on these options and let me know what looks best.” Antawn examined the water heater bids and prepared a table comparing life-​cycle costs for the three most promising options: an electric heat pump, a natural gas heater, and a solar heater (Table 21.3). Because the solar heater had an expected useful life of 20 years—​twice as long as the electric and gas options—​he decided to cost out the options for a 20-​year period. That way, he could stack the life-​cycle cost of a solar unit up against an original purchase and replacement purchase of electric or natural gas units. He used efficiency ratings and projected fuel costs to estimate energy costs over the Table 21.3  Applying Life-​Cycle Costing: Comparing Water Heater Options for 20-​ Year Period

Expected life Energy factor (EF)a Purchase price with installation Less trade-​in on present unit Replacement unit with installationc Less trade-​in Acquisition cost Energy costd Maintenance cost Less expected salvage value Life-​cycle cost

Electric heat-​pump water heater

Natural gas water heater

Solar water heater with electric backup

10 years 2.20

10 years 0.65

20 years 1.20

$3,320 −$200 $3,700

$2,050 −$260 $2,500

$9,600b −$250 NA

​−$200 $6,620 $7,600 0 ​​− $200 $14,020

​−$260 $4,030 $6,460 0 ​​− $200 $10,290

​​​NA $9,350 $3,500b $1,000 ​​− $500 $13,350

a Energy factors are based on the amount of hot water produced per unit of fuel and thereby indicate a water heater’s energy efficiency. The EF values shown here were reported by the American Council for an Energy-​Efficient Economy at www.aceee.org/​consumerguide/​waterheating.htm. b Solar water heater purchase and energy costs include electric backup for times when solar unit alone is insufficient (i.e., overcast days). c Replacement units will be needed for the electric and gas water heaters because of their shorter life expectancies. The more lightly used electric backup unit purchased for the solar option is expected to remain serviceable throughout the 20-​year period. d Energy costs are based on energy efficiency ratings, anticipated consumption, and projected fuel costs.

267

268

what does it cos t?

20-​year span. For maintenance estimates, he assumed that the electric and gas units would be replaced at about the time they began to need attention, but he knew that the solar unit would require regular servicing. “In keeping with our ‘green’ image, I was really hoping that solar power would win out, and we could move to a solar unit this time,” Antawn said, as he handed Vance his life-​cycle cost analysis. “If we hadn’t had to build in an electric backup for cloudy days, the analysis would have come out differently. The cost of the backup, plus the electricity it would need, did in the solar option. If we could have eliminated energy costs altogether and relied entirely on solar energy, it would have won.” “How do you suppose the guys on our crews would feel about cold showers every now and then?”

FROM THE ELECTRONIC TOOLKIT BOX 21.1  CALCULATING LIFE-​CYCLE COSTS Some administrators may decide to use Excel to calculate life-​cycle costs by programming the equation into a spreadsheet. First, enter the variable names or symbols (that is, acquisition cost, lifetime maintenance costs, lifetime energy costs, and salvage value) into consecutive cells of a given column. Then, enter the appropriate numerical values into cells in the adjacent column. (For directions on opening Excel and entering data refer to Box 1.1.) After the data are entered, type the equation for the life-​cycle cost using normal mathematical notations and the cell names of the data instead of symbols for the variables. For example, if the acquisition cost was entered in cell B1, lifetime maintenance costs in cell B2, lifetime energy costs in cell B3, and salvage value in cell B4, enter “=B1+B2+B3-​ B4” to calculate life-​cycle costs. Similar equations can be entered into Excel for each variable.

Variability of relevant factors This example illustrates a variety of characteristics of potential significance to the life-​cycle costs of a particular piece of equipment. For other items,

l i f e -c y c l e c o s t i n g

a different set of cost factors might be more significant. For example, a life-​cycle cost analysis for handheld devices or laptop computers for meter readers or building inspectors might be especially concerned with the durability and useful life of the units. A local government administrator making such a choice would also want to know about failure rates and service costs as well as the availability of maintenance and repair service in the event of a breakdown. Requiring the use of a carefully designed bid worksheet approved by the local government can assure that all information needed for calculating life-​ cycle costs will be uniformly secured from all bidders. Comprehensiveness and uniformity of information will permit a more direct and complete comparison of bids.

Limitations of life-​c ycle costing Although life-​cycle costing can be a useful tool for local governments, it has its limitations. The technique requires local government managers to accumulate detailed information about the various costs associated with a potential purchase. The manufacturers and sellers of equipment may supply some of that information, including energy consumption and other product performance data. Local government managers should attempt to get complete documentation from manufacturers and sellers regarding all claims made for their products. The remedies available to governments that have been deceived by false vendor claims depend on the conditions specified in the purchase contract and whether the information was presented in a deliberately misleading or inaccurate manner. Local government managers attempting to use life-​cycle costing will find that their projections can be much more accurate if they are maintaining good performance records for their own equipment. Much of the cost information related to maintenance, downtime, rental and storage charges, and other pertinent expenses may be based, in part, on the government’s previous experience with those cost factors for identical or similar equipment or on the experience of other local governments that have used the particular model in question. As such, the adequacy of the technique is linked in most cases to the availability of useful information. Life-​cycle costing can be applied to many types of local government purchases, but the information requirements and analytic effort associated

269

270

what does it cos t?

with this technique may make its application impractical in some cases. Local government managers may wish to prioritize the items that are candidates for life-​cycle costing. Finally, a frequent problem in applying life-​ cycle costing in local governments is the question of legal authority for basing purchasing decisions on life-​cycle costs rather than simply on purchase price. Several local governments, such as the city of Baltimore, Maryland, have adopted life-​cycle costing as a major component of the decision-​making process for purchasing equipment, but many others assume, often incorrectly, that they have no choice other than awarding the bid to the vendor with the lowest purchase price. In reality, many regulations specify that awards should go to the “lowest and best bid,” language that appears to leave the door open for consideration of factors other than the price tag alone. Life-​cycle costing does not eliminate the lowest bid concept; rather, it applies the concept to a greater range of costs. Local government managers who wish to use life-​cycle costing should make themselves aware of applicable legal requirements in their particular community.

BOX 21.2  OTHER APPLICATIONS OF LIFE-​CYCLE COSTING Life-​cycle costing is applicable to any purchase, but it is most practical and beneficial for large purchases, especially those involving energy-​ consuming equipment. In local governments this is likely to include computer equipment, heating and air conditioning equipment, vehicles, public works equipment, and equipment associated with water and sewage treatment, among others.

Suggested for further information Coe, Charles. Governmental and Nonprofit Financial Management. Vienna, VA: Management Concepts, 2007. Emblemsvag, Jan. Life-​Cycle Costing: Using Activity-​Based Costing and Monte Carlo Methods to Manage Future Costs and Risks. Hoboken, NJ: Wiley, 2003. Galar, Diego, Peter Sandborn, and Uday Kumar. Maintenance Costs and Life Cycle Cost Analysis. Boca Raton, FL: CRC Press, 2020.

l i f e -c y c l e c o s t i n g

League of California Cities. A Guide to Life Cycle Costing: A Purchasing Technique That Saves Money. Sacramento: League of California Cities, December 1983. Reprinted as “Life Cycle Costing.” In Practical Financial Management: New Techniques for Local Government, edited by John Matzer Jr. Washington, DC: International City Management Association, 1984. Malan, Roland M., James R. Fountain Jr., Donald S. Arrowsmith, and Robert L. Lockridge II. Performance Auditing in Local Government. Chicago: Government Finance Officers Association, 1984. See “Analysis,” 139–​167. Zemansky, Stanley D. “Life-​Cycle Cost Procurement.” In Costing Government Services: A Guide for Decision Making, edited by Joseph T. Kelley, 115–​139. Washington, DC: Government Finance Officers Association, 1984.

Web resource For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

271

22 LEASE OR BUY?1

A wide variety of equipment is needed to perform the full array of functions in a typical local government. Traditionally, much of that equipment, such as trucks, sedans, and office furniture, has been purchased by local governments; however, other types of equipment, such as computers and copy machines, often have been leased. Increasingly, consideration is being given to the relative advantages of buying versus leasing equipment. This chapter describes a technique for performing such analysis.

1 The original version of this chapter was written by Roderick C. Lee and appeared in the book’s first edition.

l e a s e o r b uy ?

Scenario: Allegheny County, West Virginia Allegheny County, West Virginia, has been leasing two brand-​new office copiers for the past four months.2 The first, a large capacity, high-​speed copier for the Sheriff’s Office, is leased by the county for $1,250 a month including maintenance. The second, a slower-​speed and lower volume copier for the Inspection and Permit Office, leases for $610 a month including maintenance. The vendor has recently offered to sell both copiers to the county. The sale price for the copier used by the Sheriff’s Office is $39,000, plus $260 per month for maintenance; the copier in the Inspection and Permit Office is available for $12,000, plus $190 per month for maintenance. Should the county continue to lease the copiers or should it buy them? To answer this question, decision makers must somehow put the series of monthly payments required to lease the copiers on an equal footing with the one-​time purchase prices and monthly maintenance fees to purchase the copiers.

Equivalent annual worth A comparison such as the one needed by Allegheny County can be developed using the equivalent annual worth method, an analytic technique for converting the up-​front purchase price of an item into a stream of uniform annual costs over the item’s useful life. This stream of uniform costs can then be compared with the costs associated with leasing the item over the same period of time. Using the equivalent annual worth method to compare the costs of leasing with the costs of purchasing is a three-​step process. First, analysts must make several assumptions regarding the item they are considering for lease or purchase. Second, on the basis of these assumptions, they apply the formula for calculating equivalent annual worth to the known cost information. Finally, they review the analysis to determine how “sensitive” the lease-​or-​buy analysis is to changes in the choices and assumptions. The

2 The scenario presented here is modeled on a Winston-​Salem, North Carolina, case study reported in A. John Vogt and Samuel H. Owen Jr., “Cost Calculation for Lease-​ Purchase Decisions,” in A Guide to Municipal Leasing, ed. A. John Vogt and Lisa A. Cole (Chicago: Municipal Finance Officers Association, 1983), 148–​152.

273

274

what does it cos t?

following discussion uses Allegheny County’s lease-​or-​buy dilemma to illustrate this three-​step process.

Making assumptions As a preliminary step in applying the equivalent annual worth method, the county’s administrators or analysts must make some assumptions about the copiers and the costs associated with owning and operating them. First, an assumption must be made about the expected useful life of each copier.Vendors can provide helpful information in that regard. Also, employees in the Sheriff’s Office and the Inspection and Permit Office can provide insights into how long copiers have routinely remained in service in their offices in the past. On the basis of information from these sources, administrators in Allegheny County assume that the copiers will continue in use for at least five more years. Second, administrators must make assumptions regarding the salvage value of the copiers at the end of five years. As with determining the length of service, information gathered from vendors and from county employees can be used to estimate salvage value. An extremely conservative assumption would be that the copiers will have no salvage value. This assumption reflects the greater risk faced by Allegheny County in buying the copiers rather than leasing them. Buying the copiers also represents a commitment to the current technology of the copiers. If new technology makes the county’s copiers obsolete, their salvage value will decline sharply. In contrast, if the county continues to lease the copiers, it can cancel the leases at the end of the term and lease newer, state-​of-​the-​art models. For this analysis, the county assumes the copiers will have no salvage value, an assumption creating a modest bias toward leasing. Third, an assumption has to be made regarding the term length of leasing and maintenance payments. To simplify this analysis, it is assumed that all periodic payments are annual. This means that the 60 monthly maintenance fees under the purchasing alternative and the 60 monthly lease payments under the leasing alternative are converted to five annual payments. This, too, creates a slight distortion in the analysis that favors leasing the copiers. The fourth assumption involves projected inflation. The advisability of a particular lease-​or-​buy decision may be influenced by projected inflation if a substantial portion of the annual cost is vulnerable to fluctuation—​for example, if the purchase price is small relative to ongoing maintenance

l e a s e o r b uy ?

expenses or if annual lease payments are not fixed for a multiyear period but, instead, are adjusted annually in accordance with an inflation index such as the Consumer Price Index.3 In the case of Allegheny County, a five-​ year, fixed-​rate lease is being offered. Accordingly, the inflation factor may be disregarded with little effect on the analysis. Finally, an interest rate must be selected to convert payments at different points in time to a comparable basis of value. There have been lengthy and conflicting discussions regarding the optimal interest rates governments should use in considering lease-​or-​buy alternatives.4 One recommendation is that the interest rate used in government applications of annual worth analyses should be the rate that the government would earn from an investment over the term of the agreement.5 In this analysis, Allegheny County administrators assume a 6 percent annual interest rate.

Calculating the equivalent annual worth On the basis of these assumptions, the county can compare its lease-​or-​buy alternatives using the formula for the equivalent annual worth method. The formula is as follows: A = P × USCRF ( for interest rate i and n years or periods), where A = an annual or periodic amount in a uniform series of such amounts in the future P = a single purchase price at the present time USCRF = uniform series capital recovery factor 3 Inflation is typically disregarded in simple annual worth analyses, as in the Allegheny County example. The assumption in doing so is that inflation will have roughly the same effect on each financing alternative. For a discussion of when and how to incorporate the effects of inflation into an annual worth analysis, see Robert E. Pritchard and Thomas J. Hindelang, The Lease/​Buy Decision (New York: AMACOM, 1980), 60–​67. 4 Vogt and Owen Jr., “Cost Calculations for Lease-​Purchase Decisions: Methods,” 139–​144. Vogt and Owen outline the major schools of thought regarding the most appropriate interest rate to use in lease-​or-​buy decisions. 5 Edward A. Dyl and Michael D. Joehnk, “Leasing as a Municipal Finance Alternative,” Public Administration Review 38, no. 6 (1978): 561–​562.

275

276

wh at doe s it cos t?

i = the interest rate for each future interest period over n periods n = the number of interest periods or like intervals in the future The uniform series capital recovery factor is an interest factor based on a formula used to convert a single amount at the beginning of a period (for example, the present) into an equivalent series of equal payments made or received during the period.6 In the Allegheny County analysis, the one-​time purchase prices for the copiers are converted to annualized purchase prices using the USCRF for 6 percent interest over a five-​year period. The USCRF can be calculated using the following formula: USCRF =

i(1 + i)n (1 + i)n − 1

where i = interest rate n = number of years or interest periods7 Because Allegheny County officials assumed 6 percent interest and a five-​ year period, their USCRF calculation was as follows: USCRF(6%,5 years) =

.06(1 + .06)5 .06 × 1.3382 0.0803 = = = 0.2374 0.3382 (1 + .06)5 − 1 1.3382 − 1

FROM THE ELECTRONIC TOOLKIT BOX 22.1  FINDING THE UNIFORM SERIES CAPITAL RECOVERY FACTOR By entering the equation for the USCRF into an Excel spreadsheet, tricky calculations can be simplified and done repeatedly for different sets of data.

6 Vogt and Owen, “Cost Calculations,” 124–​125. 7 Ibid.

l e a s e o r b uy ?

Begin by entering names or symbols (that is, interest rate and interest periods) into consecutive cells of a given column. Then, enter the appropriate numerical values in the cells in the next column. (For directions on opening Excel and entering data refer to Box 1.1.) To enter interest rates, the user must make each entry either as a decimal value or as an entry that includes the percent sign. (For example, an interest rate of 6 percent should be entered as “.06” or “6%”—​not simply as “6.”) Otherwise, Excel calculations performed under the formula will be adversely affected. To find the USCRF, enter the equation using the cell names of the data instead of letter variables. Be careful about the placement of parentheses. If the interest rate is in cell B1 and the number of years or interest periods is in cell B2, type: “=((B1*((1+B1)^B2))/​(((1+B1)^B2)-​1))” to calculate the USCRF. To calculate the equivalent annual worth, enter the purchase price into a cell, and in a separate cell of your choice specify an equation that multiplies the cell with the purchase price by the cell with the USCRF. For example if the USCRF was in cell A3 and the purchase price in cell A4, enter “=A3*A4.” After saving the spreadsheet, the calculations can be made again using different data.

Alternatively, the analyst may simply consult a USCRF table, if one is available (Table 22.1 is a limited example). Table 22.2 illustrates the application of the equivalent annual worth method to the copier in the Sheriff’s Office. The analysis reveals that the county would expect to realize annual savings of $2,621 by purchasing the copier rather than continuing to lease it. As illustrated in Table 22.3, the application of the equivalent annual worth method to the copier decision in the Inspection and Permit Office reveals an expected annual savings of $2,191 by purchasing rather than leasing.

Testing the sensitivity of the results The two analyses produce results that clearly favor the purchase of the copiers. Because the results are based in part on a set of assumptions, it is important to consider how changes in these assumptions would affect the results of the

277

278

what does it cos t?

Table 22.1  Interest Factors for Compounding Uniform Series Capital Recovery Factors Interest periods Capital recovery factors at selected rates of interest (n) 4% 5% 6%

10%

15%

1 2 3 4 5 6 7 8 9 10 15 20 25

1.1000 0.5762 0.4021 0.3155 0.2638 0.2296 0.2054 0.1874 0.1736 0.1627 0.1315 0.1175 0.1102

1.1500 0.6151 0.4380 0.3503 0.2983 0.2642 0.2404 0.2229 0.2096 0.1993 0.1710 0.1598 0.1547

1.0400 0.5302 0.3603 0.2755 0.2246 0.1908 0.1666 0.1485 0.1345 0.1233 0.0899 0.0736 0.0640

1.0500 0.5378 0.3672 0.2820 0.2310 0.1970 0.1728 0.1547 0.1407 0.1295 0.0963 0.0802 0.0710

1.0600 0.5454 0.3741 0.2886 0.2374 0.2034 0.1791 0.1610 0.1470 0.1359 0.1030 0.0872 0.0782

Source: Excerpted from “Appendix 1: Interest Factors for Compounding and Discounting,” in A Guide to Municipal Leasing, ed. A. John Vogt and Lisa A. Cole (Chicago: Municipal Finance Officers Association, 1983), 171–​198. Used by permission of the Government Finance Officers Association.

Table 22.2  Equivalent Annual Worth Analysis of the Sheriff’s Office Copier Annual cost to lease ($1,250 × 12 months) Annual cost to purchase maintenance ($260 × 12 months) annualized purchase price: ($39,000) × (USCRF for 6 percent interest and 5 years in service) = ($39,000) × (.2374) Subtotal, annual cost to purchase Expected annual savings by purchasing

$15,000 $3,120 $9,259

$12,379 $2,621

Source: Adapted from A. John Vogt and Samuel H. Owen Jr., “Cost Calculation for Lease-​Purchase Decisions,” in A Guide to Municipal Leasing, ed. A. John Vogt and Lisa A. Cole (Chicago: Municipal Finance Officers Association, 1983), 150. Adapted by permission of the Government Finance Officers Association.

analyses. In essence, how “sensitive” are the results to the assumptions (for more information on this topic see Chapter 7 “Sensitivity analysis”)? One assumption that easily can be tested for its impact is the interest rate. What if county administrators had used a 10 percent interest rate rather than the assumed 6 percent? With the higher interest rate, the equivalent

l e a s e o r b uy ?

Table 22.3  Equivalent Annual Worth Analysis of the Inspection and Permit Office Copier Annual cost to lease ($610 × 12 months) Annual cost to purchase maintenance ($190 × 12 months) annualized purchase price: ($12,000) × (USCRF for 6 percent interest and 5 years in service) = ($12,000) × (.2374) Subtotal, annual cost to purchase Expected annual savings by purchasing

$7,320 $2,280 $2,849

$5,129 $2,191

Source: Adapted from A. John Vogt and Samuel H. Owen Jr., “Cost Calculation for Lease-​Purchase Decisions,” in A Guide to Municipal Leasing, ed. A. John Vogt and Lisa A. Cole (Chicago: Municipal Finance Officers Association, 1983), 151. Adapted by permission of the Government Finance Officers Association.

annual worth of the copier used in the Sheriff’s Office over five years would be $10,288 ($39,000 × 0.2638). At the 10 percent interest rate, the equivalent annual worth of the Inspection and Permit Office’s copier would be $3,166 ($12,000 × 0.2638). While the higher interest rate makes the purchasing alternative less attractive, purchasing still would produce expected annual savings in both cases, as illustrated in Table 22.4. In this case we would say the decision here is not sensitive to the interest rate assumption, as purchasing would be the right choice even with much higher interest rates for both copiers. Another assumption that could be easily tested for its impact is the anticipated useful life of the copiers. For example, how much would the results of the analysis change if the in-​service periods for the copiers were reduced from five years to three? The shorter service life would tend to favor the leasing alternative because the purchase price would be annualized over a shorter period of time. As illustrated in Table 22.5, the three-​year period eliminates the advantage of purchasing the copier for the Sheriff’s Office and sharply reduces the purchasing advantage in the case of the Inspection and Permit Office. In contrast to the interest rate assumption, the useful life of three years would change the decision for the Sheriff’s Office copier, so this decision is sensitive to the shorter-​service-​period assumption. The inspection office copier decision is not. If the assumptions in a lease-​or-​buy analysis are weighted slightly in favor of one option (for example, lease) and still the analysis consistently

279

280

what does it cos t?

Table 22.4  Testing the Sensitivity of Interest Rates for Equivalent Annual Worth Analysis Sheriff’s Office copier Annual cost to lease At 6 percent interest rate maintenance annualized purchase pricea annual cost to purchase expected annual savings by purchasing At 10 percent interest rate maintenance annualized purchase pricea annual cost to purchase expected annual savings by purchasing

Inspection and Permit Office copier

$15,000

$7,320

$3,120 9,259 $12.379

$2,280 2,849 $5,129

$2,621

$2,191

$3,120 10,288 $13,408

$2,280 3,166 $5,446

$1,592

$ 1,874

Source: Adapted from A. John Vogt and Samuel H. Owen Jr., “Cost Calculation for Lease-​Purchase Decisions,” in A Guide to Municipal Leasing, ed. A. John Vogt and Lisa A. Cole (Chicago: Municipal Finance Officers Association, 1983), 150–​151. Adapted by permission of the Government Finance Officers Association. a For five-​year in-​service period.

Table 22.5  Testing the Sensitivity of In-​Service Periods for Equivalent Annual Worth Analysis

Annual cost to lease At five-​year in-​service period maintenance annualized purchase pricea annual cost to purchase

Sheriff’s Office copier

Inspection and Permit Office copier

$15,000

$7,320

$3,120 _​9,259 $12,379

$2,280 _​2,849 $5,129

expected annual savings by purchasing At three-​year in-​service period maintenance annualized purchase pricea annual cost to purchase

$2,621

$2,191

$3,120 _​14,590 $17,710

$2,280 _​4,489 $6,769

expected annual savings (loss) by purchasing

−$2,710

$551

Source: Adapted from A. John Vogt and Samuel H. Owen Jr., “Cost Calculation for Lease-​Purchase Decisions,” in A Guide to Municipal Leasing, ed. A. John Vogt and Lisa A. Cole (Chicago: Municipal Finance Officers Association, 1983), 150–​151. Adapted by permission of the Government Finance Officers Association. a With 6 percent interest.

l e a s e o r b uy ?

favors the other option (for example, purchase), the result may be taken as a clear endorsement of the second option. More ambiguous results are less easily interpreted. In addition, less quantifiable factors, such as losing the opportunity to lease updated state-​of-​the-​art equipment, must be weighed against projected savings to reach a satisfactory decision. In the case of Allegheny County, the analysis was most sensitive to assumptions regarding the useful life of the equipment. Local officials would be well advised to reassess their confidence in a five-​year lifespan for copiers before committing to their purchase.

BOX 22.2  OTHER APPLICATIONS OF LEASE-​BUY ANALYSIS Lease-​buy analysis has broad potential applicability for local government decision making. Cities and counties sometimes make mode-​ of-​acquisition decisions almost out of habit for items ranging from heavy-​ duty road maintenance equipment to office furnishings. For example, most local governments purchase their police and motor pool fleets without seriously considering the leasing option, and most make office building and other facility decisions without fully considering the alternatives. Better informed choices could be made by incorporating lease-​buy analysis into the process.

References Dyl, Edward A., and Michael D. Joehnk. “Leasing as a Municipal Finance Alternative.” Public Administration Review 38, no. 6 (1978): 557–​562. Pritchard, Robert E., and Thomas J. Hindelang. The Lease/​ Buy Decision. New York: AMACOM, 1980. Vogt, A. John, and Samuel H. Owen Jr. “Cost Calculations for Lease-​Purchase Decisions: Methods” and “Cost Calculations for Lease-​Purchase Decisions: Five Case Studies.” In A Guide to Municipal Leasing, edited by A. John Vogt and Lisa A. Cole, 115–​145, 147–​170. Chicago: Municipal Finance Officers Association, 1983.

281

282

what does it cos t?

Suggested for further information Bierman, Harold, Jr., and Seymour Smidt. Advanced Capital Budgeting: Refinements in the Economic Analysis of Investment Projects. New York: Routledge, 2007. Bierman, Harold, Jr., and Seymour Smidt. The Capital Budgeting Decision: Economic Analysis of Investment Projects, 9th ed. New York: Routledge, 2007. See Chapter 14. Coe, Charles. Governmental and Nonprofit Financial Management. Vienna, VA: Management Concepts, 2007. Contino, Richard M. The Complete Equipment-​ Leasing Handbook: A Deal Maker’s Guide with Forms, Checklists, and Worksheets, 2nd ed. York House Press, 2015. The Economist Numbers Guide: The Essentials of Business Numeracy, 6th ed. New York: Public Affairs, 2014. See pp. 58–​59.

Web resource For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

23 COST-​EFFECTIVENESS ANALYSIS A TRUNCATED VERSION OF COST-​BENEFIT ANALYSIS

All managers would like to think that the benefits derived from their programs exceed the costs. Furthermore, conscientious managers want to believe that the approach they have adopted to deliver a particular program provides a more favorable return relative to costs than would any other approach. Only rarely, however, are the costs and benefits associated with program alternatives—​ even the most obvious ones—​identified and analyzed in a systematic fashion. This chapter introduces cost-​effectiveness analysis, a truncated version of cost-​benefit analysis that offers a practical decision-​making tool applicable to cases where either of two conditions exists: 1. 2.

the costs of all alternatives are equal, allowing the analyst to focus on differences in benefits among the alternatives the benefits of all alternatives are equal—​or may be presumed to be equal—​allowing the analyst to focus on differences in cost

284

what does it cos t?

Cases that do not meet these conditions—​that is, cases in which one policy or program option carries costs and benefits that are complex and differ from the costs and benefits of other options—​require the more complicated analytic technique of cost-​benefit analysis.

Cost-​b enefit and cost-​e ffectiveness analyses Although the focus here is on the simpler technique of cost-​effectiveness analysis, a few of the fundamentals of cost-​benefit analysis will be explored first. Cost-​benefit analysis in its most sophisticated form can be quite complex. It is a technique designed to weigh all of an action’s costs against all of its benefits to assist decision makers in their choices. Part of the technique’s complexity lies in the difficulty of identifying all costs and benefits associated with a given program or decision. Part lies in the difficulty of quantifying costs and benefits on a common or comparable scale or metric.1 This typically requires monetizing all costs and benefits. Furthermore, the analysis must account for the effects of time in comparing costs and benefits. Most easily recognized are the costs and benefits that are both tangible and direct (for example, labor, equipment, and energy costs associated with a given program and the value of a particular product). More difficult to identify and measure are intangible direct costs and benefits, indirect costs and benefits, and pecuniary costs and benefits. Indirect costs and benefits, often referred to as spillover effects or, in the jargon of cost-​benefit analysis, as externalities, are often unanticipated, difficult to identify, and hard to measure. An example of a negative spillover or externality would be odors coming from a city’s sewage treatment plant. A positive spillover might be gains in the value of commercial property following a municipality’s decision to spend money on sidewalk improvements and beautification in a downtown area. Comprehensive cost-​benefit analysis addresses each of these elements and provides decision makers with a method of assessing the return on a particular investment or comparing the relative advantages of a variety of 1 Warren S. Eller, Brian J. Gerber, and Scott E. Robinson, Public Administration Research Methods:Tools for Evaluation and Evidence-​Based Practice, 2nd ed. (New York: Routledge, 2018), 398. See also “Applied Decision Tools,” Chapter 19, 395–​420.

cos t -e ffec t i v e ne s s a na ly s i s

options. Program options may be judged, for example, on net benefit (the difference between benefits and costs) or benefit-​to-​cost ratios. Many administrative reports in local government are incorrectly portrayed by their authors as “cost-​benefit analysis” when actually they fall short of the rigor required by this technique. Analysts conducting true cost-​ benefit analysis must sort out and quantify tangible costs and benefits to the local government, service recipients, and the broader society for each alternative; intangible costs and benefits; and net social benefits. Relevant considerations may include elasticity of demand and supply, pecuniary effects, shadow pricing, and redistribution effects. Readers confronting cases where neither the costs nor the benefits of a set of options are equal are encouraged to consult any of the more extensive texts on the subject of cost-​benefit analysis for more thorough instruction. For the purposes here, however, the focus is on the analysis of options with (a) equivalent price tags, where the analyst seeks the alternative that will produce greatest benefits, or (b) equivalent benefits, where the search is for lowest cost. In either of these cases the related though simpler technique of cost-​ effectiveness analysis is appropriate.

BOX 23.1  ANALYZING THE DIFFERING BENEFITS OF EQUAL-​COST ALTERNATIVES One version of cost-​effectiveness analysis pits equal-​costing alternatives against one another to see which one produces the greatest benefits. Local governments confront this circumstance whenever a set amount of funds is appropriated in hopes of making progress toward a purpose of large scale. Consider, for example, a $50,000 appropriation for new summer recreation programs in the parks, a first-​time appropriation of $200,000 for economic development, or a $1 million appropriation to upgrade the condition of city streets. In each instance, local government officials would be expected to find the alternative that will produce the greatest benefit within the constraints of the appropriation. With $1 million to spend on streets, should the city do major reconstruction work on the roadways in the worst condition or should it do less extensive maintenance and repair work on a larger portion of the street inventory? Analysis by the city of Portland, Oregon, showed how

285

286

what doe s it c os t?

much further $1 million would go doing preventive sealing as opposed to doing reconstruction (Figure 23.1). If city officials opt to spend their money on preventive measures rather than street reconstruction, should they consider only the number of lane-​miles that could be treated using various alternatives? Should they not also consider the effect each type of treatment would have on extending pavement life? Portland’s analysis shed light on that topic, too (Table 23.1). A thorough cost-​effectiveness analysis would take both factors into account.

Figure 23.1  Number of Lane Miles of Roadway That Can Be Treated with $1 Million, Using Various Treatment Options Notes: (1) Based on annualized cost per lane mile and average life gained by various treatment types, available in a 2005 report to the Transportation Research Board by Larry Galehouse, Helen King, David Leach, Jim Moulthrop, and Bill Ballou, “Preventive Maintenance Treatment Performance at 14 Years,” 2005, 19. (2) Exact costs of each treatment would vary in the City of Portland, as would the number of miles that could be treated by each treatment type. Nevertheless, preventive maintenance allows street departments to treat more miles than more expensive rehabilitation and reconstruction. Source: City of Portland, “Street Paving: More Proactive Maintenance Could Preserve Additional City Streets Within Existing Funding” (Portland, OR: Office of the City Auditor, July 2006), 18.

cos t- e ffec t i v e n e s s a n a ly s i s

Table 23.1 Years of Pavement Life Gained Through Various Preventive Treatments Treatment

Estimated pavement life extension, years

Crack filling Fog seal Seal coat Chip seal Double chip seal Slurry seal Microsurfacing Thin (1.5 inch) hot mix Hot-​mix overlay (1.5 inch), after milling

2 to 6 1 to 4 3 to 6 3 to 7 7 to 10 3 to 7 3 to 9 5 to 10 2 to 12

Sources: City of Portland, Oregon, Street Paving: More Proactive Maintenance Could Preserve Additional City Streets Within Existing Funding (Portland, Oregon: Office of the City Auditor, July 2006), 22. Information based on Ann Johnson, Best Practices Handbook on Asphalt Pavement Maintenance (Minneapolis: University of Minnesota, Minnesota Technology Transfer Center, February 2000), 17; Gary Hicks, Stephen Seeds, and David Peshkin, Selecting a Preventive Maintenance Treatment for Flexible Pavements, Report no. FHWA-​IF-​00-​027 (Washington, DC: Federal Highway Administration; Foundation for Pavement Preservation, 2000), 9; Larry Galehouse, Helen King, David Leach, Jim Moulthrop, and Bill Ballou, Preventive Maintenance Treatment Performance at 14 Years (Washington, DC: Transportation Research Board, 2005), 19; Larry Galehouse, “Strategic Planning for Pavement Preventive Maintenance,” in Pavement Preservation Compendium, Report no. FHWA-​IF-​03-​21 (Washington, DC: Federal Highway Administration, 2003), 8; and D.G. Perkins, T.E. Hoerner, and K.A. Zimmerman, Optimal Timing of Pavement Preventive Maintenance Treatment Applications, NCHRP Report no. 523 (Washington, DC: Transportation Research Board of the National Academies, National Cooperative Highway Research Program, 2004), 10–​13.

Scenario: Surf City, Florida Surf City is a small Florida town of 12,000 people. Residents enjoy a broad range of municipal services, including refuse collection. Christopher Hogan, the sanitation director of Surf City, has long taken pride in his small operation. For several years he has provided both residential and commercial refuse collection services, as well as special brush pickups, with a department consisting of only nine refuse collection staff members.Two three-​person crews handle most of the residential collections, and the other three collectors provide commercial dumpster service, brush collection, and residential backup, as needed. Hogan used to think that he had a pretty lean staff, but recently he has begun to question that

287

288

what does it cos t?

assumption. Over the last year and a half a parade of vendors has marched through his office singing the praises of different equipment options and reduced crew sizes. “I’ve always been skeptical of the claims of salesmen,” Hogan confided to his secretary following yet another vendor’s spiel, “but several cities in the region have gone to two-​person and even one-​person crews, apparently with good results. Most of the salesmen have been downplaying the cost of their equipment and emphasizing reduced labor costs. I’m going to try to sort out all the major costs and benefits for various crew sizes and equipment types and see what makes the most sense.”

Direct costs and benefits of refuse collection options Chris Hogan identified four equipment and employee configurations that he considered to be practical for residential refuse collection based on his own experience, the claims of various equipment vendors, and reports from sanitation directors in other communities. Although the costs varied from one option to another, all four would deliver comparable services or benefits—​making cost-​effectiveness analysis an appropriate technique for this case. The four alternatives were: • • • •

three-​person crews using rear-​loading trucks (Option A, the current system) two-​person crews using rear-​loading trucks (Option B) two-​person crews using side-​loading trucks (Option C) one-​person crews using side-​loading trucks (Option D)

Each option addressed only basic residential refuse collection. Additional arrangements would have to be made for servicing businesses and for collecting brush and other yard waste. The first alternative for residential refuse collection stood as the baseline against which the other alternatives could be compared. Currently, two three-​person crews use rear loaders to cover all residential routes in Surf City. Three other drivers handle commercial service and brush collection, as well as provide backup coverage for the residential crews. Because this is the system now in place, its feasibility has been well established (see Table 23.2).

co s t -e ffe c t i v e n e s s a n a ly s i s

Table 23.2  Equipment and Crew Requirements of Four Refuse Collection Options Option A

Option B

Option C

Option D

Crew 1

Residential route rear loader 1 equipment operator 2 laborers

Residential route rear loader 1 equipment operator 1 laborer

Residential route side loader 1 equipment operator 1 laborer

Residential route side loader 1 equipment operator

Crew 2

Residential route rear loader 1 equipment operator 2 laborers

Residential route rear loader 1 equipment operator 1 laborer

Residential route side loader 1 equipment operator 1 laborer

Residential route side loader 1 equipment operator

Crew 3

Commercial route, brush, and residential backup front loader 1 equipment operator

Commercial route front loader 1 equipment operator

Residential route side loader 1 equipment operator 1 laborer

Residential route side loader 1 equipment operator

Crew 4

Commercial route, brush, and residential backup front loader 1 equipment operator

Commercial route front loader 1 equipment operator

Commercial route front loader 1 equipment operator

Commercial route front loader 1 equipment operator

Crew 5

Commercial route, brush, and residential backup rear loader 1 equipment operator

Brush collection and residential backup rear loader 1 equipment operator

Commercial route front loader 1 equipment operator

Commercial route front loader 1 equipment operator

Crew 6

None

None

Brush collection rear loader 1 equipment operator

Brush collection and residential backup rear loader 1 equipment operator

Total

Equipment 3 rear loaders 2 front loaders 0 side loaders Personnel 5 equipment operators 4 laborers

Equipment 3 rear loaders 2 front loaders 0 side loaders Personnel 5 equipment operators 2 laborers

Equipment 1 rear loader 2 front loaders 3 side loaders Personnel 6 equipment operators 3 laborers

Equipment 1 rear loader 2 front loaders 3 side loaders Personnel 6 equipment operators 0 laborers

289

290

what does it cos t?

The premise of the second alternative is that two workers should be able to do the work that previously required three, with little, if any, modification of equipment. The basis for such a premise is normally that workers have been underutilized either because they have been required to exert less than a full day’s effort or because of deployment problems, such as imbalanced routes. Hogan had his doubts about this alternative (Option B), but decided to include it anyway. The third alternative (Option C) trims the size of individual crews by means of revised equipment configurations offering labor-​saving features. Side-​loader trucks equipped with right-​side steering from a curbside compartment make it more practical for the driver to help load the truck. This option accommodates the smaller crew size by reducing the length of the routes and adding a third residential refuse collection crew. With the addition of this crew, the total number of personnel is nine—​the same as the current system’s total. The fourth option (Option D) takes the third alternative one step further. The driver-​collector becomes solely responsible for the collection of the refuse route. In such instances the side-​loader vehicle is equipped with a hydraulic lift to partially or fully mechanize the system and make one-​ person operation feasible. Option D restores the backup crew availability that is eliminated in Option C. After perusing a few books and articles on the use of cost-​benefit analysis and a few others on cost-​effectiveness analysis, Hogan was satisfied that he could apply the simpler of the two techniques to the analysis of his options. Because the benefits of the four options were virtually equal (that is, the collection of all the community’s residential and commercial refuse, as well as brush), this case qualified for cost-​effectiveness analysis. Using cost-​effectiveness analysis, he could concentrate on differences in cost among the alternatives. Although Hogan understood the potential significance of indirect costs, intangible costs, and even pecuniary costs, he decided as a matter of expedience that he would keep in mind the possible existence of such factors, try to identify the most important ones, and attempt to take them into account in deciding a close call between two alternatives. He would not, however, try to quantify such factors. He would focus instead on identifying and quantifying major direct costs. The results of his effort to tabulate direct costs are shown in Table 23.3.

co s t -e ffe c t i v e n e s s a n a ly s i s

Table 23.3  Projected Annual Direct Costs of Four Refuse Collection Options Option A Standard mode of operation for residential collection Personnel for residential, commercial, and special solid waste collection

Costs Wagesa Equipment operators ($) Laborers ($) Overtime ($) FICAb ($) Insurancec ($) Retirement contributiond ($) Uniformse ($) Supplies ($) Equipment Annualized capital costf Rear loaders ($) Side loaders ($) Front loaders ($) Collection cartsg ($) Operation and maintenance ($) Landfill fees ($) TOTAL

Option B

Option C

Option D

rear loader rear loader side loader side loader with 3-​ with 2-​ with 2-​ with 1-​ person crew person crew person crew person crew 9 total—​two 7 total—​two 9 total—​three 6 total—​three 3-​person 2-​person 2-​person 1-​person residential residential residential residential crews; 3 crews; 3 crews; 3 crews; 3 persons for persons for persons for persons for commercial, commercial, commercial commercial, brush and brush and and brush; brush and backup backup no backup backup

189,100

189,100

223,800

239,400

126,000 2,500 24,138 31,510 25,408

63,000 2,500 19,350 25,210 20,368

94,500 2,500 24,381 31,830 25,664

0 2,500 18,384 23,940 19,352

5,400 6,000

4,200 6,000

5,400 6,000

3,600 6,000

88,350 0 54,150 52,174 59,000

88,350 0 54,150 52,174 62,000

29,450 99,750 54,150 52,174 65,000

29,450 99,750 54,150 52,174 65,000

324,000 $987,730

324,000 $910,402

324,000 $1,038,599

324,000 $937,700

a Based on annual wages of $34,700 for an equipment operator on a two-​or three-​person crew, $39,900 for an equipment operator on a one-​person residential route crew, and $31,500 for a laborer. b Budgeted at 7.6 percent of wages and overtime. c Estimated at 10 percent of wages. d Budgeted at 8 percent of wages and overtime. e Budgeted at $600 per employee. f See Table 23.4. g Based on the assumption that carts are bought for all 5,217 residential collection customers.

291

292

what does it cos t?

Table 23.4  Annualized Capital Costs for Refuse CollectionVehicles and Collection Carts Equipment

Purchase price

Anticipated useful life

Projected salvage valuea

Annualized capital costb

Rear loader (for residential service) Side loader (for residential service) Front loader (for commercial dumpster service) Collection carts

$186,000

  6 years

$9,300

$29,450

$210,000

  6 years

$10,500

$33,250

$171,000

  6 years

$8,550

$27,075

$120

10 years

$0

$12

a Assumes 5 percent salvage value at the end of six years for vehicles but no salvage value for carts. b Based on straight-​line depreciation (see Chapter 18).

Major costs associated with all four options center on employees, equipment, and landfill fees. Each option requires a different number of employees, thereby affecting the budget for wages, benefits, and uniforms. Each requires at least five equipment operators, while the number of laborers ranges from none to four. In recognition of the greater individual responsibility associated with one-​person operations, Hogan included a 15 percent wage increase for equipment operators assigned that duty. Operating expenses for equipment include maintenance, fuel, and associated supplies. Annualized capital costs for equipment, based on costs for different types of vehicles, a six-​year projected life, and 5 percent salvage value, are shown in Table 23.4. New collection carts would be required for the two options using side loaders (options C and D) to facilitate lifting, but Hogan decided to include carts with all of the options as a needed improvement in level of service and convenience. (For annualizing costs of capital items, see Chapter 18 for a description of straight-​line depreciation.) Projected landfill fees do not vary from one option to another. The total tonnage of solid waste collected and disposed will not be affected by the choice of one option over another. Viewed on a community-​wide basis, the benefits of each of the alternatives would include the value of refuse collection services to service recipients individually (removal of their garbage) and collectively (community appearance and public health). This analysis assumes that these benefits would be secured equally from each of the four options. If the benefits, like the costs, had varied from one option to another, Hogan would not have

co s t -e ffe c t i v e n e s s a n a ly s i s

Table 23.5  Projected Annual Direct Costs of Four Refuse Collection Options Standard mode of operation for residential collection

Option A

Option B

Option C

Option D

Rear loader with three-​person crew

Rear loader with two-​person crew

Side loader with two-​person crew

Side loader with one-​person crew

Benefits Costs Net benefitsa Benefit-​to-​cost ratiob

$998,500 $987,730 $10,700 1.0109

$998,500 $910,402 $88,098 1.0968

$998,500 $1,038,599 −$40,099 0.9614

$998,500 $937,700 $60,800 1.0648

a Net benefit is the difference between benefits and costs. This is the “return on investment” in actual dollars rather than as a ratio or “rate of return.” b The benefit-​to-​cost ratio is calculated by dividing benefits by costs. The result shows the “return on investment” as a ratio that may be converted easily to “rate of return” or percentage gain or loss. In this example, the benefits of Option D are projected to exceed costs by approximately 6.48 percent.

been able to apply cost-​effectiveness analysis to this problem. With cost-​ effectiveness analysis, one or the other—​either costs or benefits—​must be stable across all options, allowing the analysis to focus on the one that varies. If both costs and benefits had varied from one option to another, the more sophisticated cost-​benefit analysis would have been required. The residents and businesses of Surf City pay refuse collection fees totaling $998,500. For his analysis, Hogan used this amount to approximate the value—​or benefit—​of the refuse collection service. The results of the analysis indicated advantages for two of the three alternatives when compared with the current three-​person rear-​loading system, advantages that are attributable primarily to reduced crew sizes (see Table 23.5). Option C was the only option having a negative net benefit—​that is, costs exceeding the $998,500 benefit assumed for all four options. Although Option B showed the greatest potential gain in net benefits, Hogan remained skeptical about whether two-​person crews could actually complete their work on schedule with current equipment and route lengths. As he promised himself, Hogan attempted to consider some of the indirect and intangible costs and benefits, although without trying to quantify them precisely. On this score, he had further doubts about Option B. Side-​loading trucks are designed for driver involvement in the loading process; rear loaders are not. Without a side-​loading truck, would a driver moving to the rear of the vehicle and climbing into and out of the cab be more accident prone? Would the two-​member crew be more injury prone than the current three-​member crew or even a one-​person

293

294

what does it cos t?

crew relying on a hydraulic lift? Would a two-​person crew on a route previously handled by a three-​member crew be forced to cut corners to keep pace, resulting in more spillage, more complaints, and a host of negative externalities? This analysis convinced Hogan that it was time for the current system to give way to a new approach, but his reservations about Option B caused him to prefer Option D, despite its projected higher cost compared to B. By shifting to one-​person side loaders for the residential routes, Hogan figured he could reduce route length, add a truck to the current configuration, retain backup services, probably reduce back injuries, and save more than $50,000 per year compared to the current system. He thought he could persuade the city manager that Option D was the way to go.

Postscript The techniques associated with cost-​benefit and cost-​effectiveness analyses provide a useful basis for sorting out difficult choices. These techniques may be especially revealing when applied rigorously and on a comprehensive basis—​and somewhat less useful when applied less rigorously or less comprehensively. Public managers conducting cost-​effectiveness analysis—​and especially those conducting the more rigorous cost-​benefit analysis—​must be realistic not only about the time required and the complexity of this process but also about their expectations regarding the impact of analysis. They must also recognize their obligation to disclose the assumptions embedded in their analysis as well as the possible weaknesses introduced by any procedural shortcuts. A realistic expectation for cost-​effectiveness or cost-​benefit analysis in public decision making is that it can help to objectively inform a process that is, and will continue to be, largely political in nature. Greater thoroughness often strengthens the influence of an analytic effort, but even the most comprehensive and sophisticated analysis might not have the final word. In the case of Surf City, several assumptions included in the calculations or omitted from explicit consideration in that analysis should be mentioned by the sanitation director in his presentation to decision makers. For example, he chose a life expectancy of six years for all trucks and a 15 percent pay

co s t -e ffe c t i v e n e s s a n a ly s i s

differential for the operators of one-​person vehicles. Higher or lower figures could influence the results of the analysis.2 Decision makers should also be informed about any important indirect or intangible factors relevant to the refuse collection considerations. For example, after conferring with the streets superintendent, Sanitation Director Hogan confidently projects a reduction in curb damage if the city switches to one-​person side loaders with smaller wheelbases than the current equipment. Furthermore, Hogan anticipates a reduction in employee injuries, especially in instances where the trucks are equipped with hydraulic lifts. Of extreme importance with regard to successful implementation is anticipated employee morale. That factor, which in part prompted the proposed wage adjustment for one-​person collection crews, should be addressed in the deliberations. Finally, feasibility issues—​technical and political—​associated with each alternative should be considered. Are the projected advantages of a one-​ person operation realistic? Will such an operation work in Surf City? Is too much on-​street parking permitted to allow the side loader to maneuver close enough to the curb to collect garbage with its mechanical arm? Will new policies affecting residents, such as the required use of special roll-​out containers or plastic bags or restricted on-​street parking, be necessary? Will residents accept such policies or resist them by political means? Will employee groups support the alternatives to secure better wages or oppose them because they reduce employment opportunities? Would strong employee resistance destroy the viability of an alternative? A variety of specific factors influence the organizational and political environment of a jurisdiction. Although many will be difficult to quantify, they should nevertheless be considered in some fashion along with their more easily quantified counterparts.

2 Performing the computations a few additional times using different numbers for vehicle depreciation or pay levels or substituting slightly different figures for other key assumptions (a process described in Chapter 7 and known as sensitivity analysis) reveals how sensitive the analysis is to such decisions or to possible errors in various assumptions. Analysts are wise to take that additional step.

295

296

what does it cos t?

Reference Eller, Warren S., Brian J. Gerber, and Scott E. Robinson. “Applied Decision Tools.” In Public Administration Research Methods: Tools for Evaluation and Evidence-​Based Practice, 2nd ed. New York: Routledge, 2018.

Suggested for further information Boardman, Anthony, David Greenberg, Aidan Vining, and David Weimer. Cost-​ Benefit Analysis: Concepts and Practices, 4th ed. Cambridge, UK: Cambridge University Press, 2018. Cellini, Stephanie Riegg, and James Edwin Kee. “Cost-​Effectiveness and Cost-​ Benefit Analysis.” In Handbook of Practical Program Evaluation, 4th ed., edited by Kathryn E. Newcomer, Harry P. Hatry, and Joseph S. Wholey, 636–​672. Hoboken, NJ: Jossey-​Bass, 2015. Hollands, Fiona, A. Brooks Bowden, Clive Belfield, Henry M. Levin, Henan Cheng, Robert Shand, Yilin Pan, and Barbara Hanisch-​Cerda. “Cost-​ Effectiveness Analysis in Practice: Interventions to Improve High School Completion.” Educational Evaluation and Policy Analysis 36, no. 3 (2014): 307–​326. Local Government Association. Prevention: A Shared Commitment—​Making the Case for a Prevention Transformation Fund. London, UK: Local Government Association, 2015. www.local.gov.uk Michel, R. Gregory. Decision Tools for Budgetary Analysis. Chicago: Government Finance Officers Association, 2001. See especially pp. 77–​82. Mishan, E.J., and Euston Quah. Cost-​Benefit Analysis, 6th ed. New York: Routledge, 2021. Wittmer, Dennis P. and Robert P. McGowan. “Five Conceptual Tools for Decision Making.” In Handbook of Public Administration, 3rd ed., edited by Jack Rabin, W. Bartley Hildreth, and Gerald J. Miller, 315–​344. Boca Raton, FL: CRC/​Taylor & Francis, 2007.

Part V PROCESS IMPROVEMENT

Once established, operating procedures often remain in place for a very long time. Even practices established informally or to meet a specific temporary need can become locked in place. Procedures that were simply the personal preferences of an early operator become the expected pattern for all successors. Modifications may be tacked onto or spliced into existing procedural guidelines as needs arise. New responsibilities undertaken by a work unit are not always blended carefully into current procedures or assigned to particular employees as part of some logical strategy. Instead, the new task often is added to the duties of an individual who is thought to be qualified to do it, who has the time for it, or who has not received a new assignment as recently as other coworkers. The haphazard nature of many evolving procedures is a recipe for problems. Employees may come and go, but procedures and workload distributions—​even when poorly devised—​often remain. Managers may be so accustomed to their operation as they inherited it or as it has evolved that they fail to see procedural inefficiencies or the misallocation of work duties; they simply assume that their work units are functioning appropriately. They recognize persistent problems, but they have always had those problems and they deal with them as best they can. They assume that this is all they can do.

298

process imp ro v em ent

Techniques of problem identification—​ Pareto charts and cause-​ and-​ effect diagrams—​can be helpful in sorting out the causes of problems in a process or program and designing a strategy for dealing with the worst of them. Techniques of work-​flow analysis—​such as process flow charting—​can help managers see any weaknesses or procedural flaws in their operations more objectively. The use of control charts can help them recognize important signals in performance fluctuations, distinguishing these signals from the routine patterns of performance ups and downs.

24 PARETO CHARTS

Does it ever seem that 20 percent of the tasks you perform cause 70 or 80 percent of your headaches or that just two or three of your friends create 90 percent of the drama in your life? If so, Vilfredo Pareto, the esteemed Italian economist, might have predicted it. In the late 1800s, Pareto studied the distribution of income and wealth in Italy and found that 80 percent of the land was owned by 20 percent of the population. Looking further, he discovered similar ratios in other countries. Relatively small percentages of the populations in these countries accounted for disproportionately large shares of land and wealth. Following Pareto’s study of land and wealth, other observers have noticed similarly disproportionate relationships in other aspects of society. Today, when a company finds that an overwhelming share of its revenues comes from a relatively small slice of its clients, that most of its sales are made by a small portion of its sales force, or that most of its product defects are linked to only a few recurring causes, it might chalk each of these up to the Pareto Principle. The Pareto Principle—​more popularly known as the 80/​20 rule—​suggests that 80 percent of the consequences come from 20 percent of the causes.1 1 The principle was actually first suggested by management thinker and consultant Joseph Juran, who applied Pareto’s findings on the distribution of wealth to other realms broadly and attached Pareto’s name to the concept.

300

process imp ro v em ent

Although hardly scientific or perfectly predictive, the principle is more a rule of thumb than a law of nature. It calls attention to the frequent occurrence of disproportionate ratios of causes and consequences, which can be found in many aspects of our personal and professional lives. A management tool called the Pareto chart is of particular interest to performance improvement enthusiasts. These charts are useful in identifying instances in which a small proportion of cases is causing disproportionate problems or expenses. They can help managers and analysts target their efforts as they work toward solutions.

Scenario: Fiddler, Tennessee Tom McGraw, the public works director for the city of Fiddler, was frustrated. He was fed up with the growing volume of complaints over the performance of public works crews. Tom did not field most of the complaints—​he had staffers who took most of those calls—​but he saw the monthly totals in the department’s performance report and more than a few complainants had spoken to him directly. Two months ago, he heard about crew members just standing around at job sites or even sitting and visiting rather than working. Last month, a couple of friends stopped him as he was leaving church to tell him that his crews were more than three weeks behind schedule in removing leaves from the curbs in their neighborhoods in the annual fall cleanup. Recently, at his barbershop he heard about pothole problems. He was sure it would be something else next month. He had spoken repeatedly over the past several months with operations supervisor Tyler Swift, who always promised to take care of the problems. But the problems persisted and the number of complaints for the past year totaled 1,582, averaging about six complaints per working day. Tom asked his administrative assistant Sonja Twain to meet with him in his office to discuss the problem. “The complaints are on the rise,” Tom said. “I am concerned that it’s about to get away from us. We need to turn this around quickly.” “What can I do to help?” Sonja asked. “Well first, we need to figure out where to attack this problem,” Tom replied. “Are most of the complaints about idle crews, fall cleanup, pothole problems or something else? Those are the things that I hear about, but

pa re t o c h a rt s

I don’t hear most of the complaints myself and our complaint statistics are consolidated into a single line. I need for you to break this out for me, so we can be sure that we are working on the things that will make the biggest difference in the overall problem.” “What does Tyler say about this?” Sonja asked, referring to the operations supervisor. “I guess you have spoken to him about this subject.” “Oh, yes, many times,” Tom replied. “Tyler sings the same tune every time. It’s always about the crews—​their lousy work ethic, general laziness, need for constant supervision. His solution is to stay on them and push them harder. That goes as his solution, whatever the specific complaint may be.” “I will get on this and get back to you as soon as I can,” Sonja promised.

Pareto charts Analytic time and talent as well as management attention is too valuable to waste it on unimportant problems or unimportant causes.2 Resources directed toward the systematic analysis and careful redesign of operations should be allocated to the most important problems and their most relevant causes. Minor problems and minor causes can be treated later, if at all. Pareto charts can help sort all of this out. According to the rule of thumb known as the Pareto Principle a few key factors are likely to disproportionately influence favorable or unfavorable results or conditions. These are what Joseph Juran labeled the “vital few.” Does the county warehouse have an inventory problem? Odds are that 80 percent of the inventory outages occur among only 20 percent of the inventory items. Does the city have a crime problem? We might find that most of the offenses fall into a small set of crime categories or that a relatively small number of groups and individuals are responsible for a disproportionately large number of them. A good strategy would be to concentrate on those categories or on those groups and individuals. A Pareto chart for the inventory problem would be a bar chart showing the number of unfilled orders for each of the items most commonly experiencing outages. The bars would be arranged in descending order with the 2 Stephan Konz and Steven Johnson, Work Design: Occupational Ergonomics, 7th ed. (Boca Raton, FL: CRC Press, 2016).

301

302

process imp ro v em ent

most problematic item on the left. A line graph at the top of the chart would show the cumulative percentage of unfilled orders—​for example, showing that the first bar constituted 45 percent of all unfilled orders, that the first two bars together constituted 75 percent of all unfilled orders, and so forth. Similarly, a Pareto chart for the crime problem might show that the first three bars, representing the three most frequently committed crimes, constituted 60 percent of all crimes. Creating a Pareto chart is simple. Some authorities have suggested that it is especially easy if one begins with a three-​column table.3 The first column lists the relevant categories of a given problem. These might be the inventory items subject to outages, the types of crimes, or the instigators of crimes. Alternatively, the first column might list major categories of complaints, categories of service failures, or other types of problems. The second column lists the frequencies of occurrence of the specified problem for each of the categories listed in the first column. The third column shows the percentage of the whole represented by each category. Next, the table will be sorted from the most common occurrence to the least. Then, the conversion to a descending-​order bar chart with a cumulative percentage line graph superimposed over the bar chart will be simple. These are the steps for constructing a Pareto chart: 1. Create a bar chart showing the magnitude of a given problem.4 Each bar represents a category relevant to the problem and the bar’s height depicts the frequency of the problem associated with that category. For example, the bars could be categories of crime with the height of each bar depicting the number of crimes committed in that category. 2. Arrange the bars in descending order with the tallest bar on the left. If one of the bars serves as a “miscellaneous” or “other” category, it should be placed at the far right and often is the only exception to the descending order of bars. 3. Adjust the overall height of the figure to be tall enough to accommodate a cumulative bar stacking all individual bars on top of one another. 3 Kaoru Ishikawa, Guide to Quality Control (Tokyo, Japan: Asian Productivity Organization, 1982), 43–​44. 4 A Pareto chart can also show a favorable condition—​for example, sales by members of the sales force or revenues from different customers. But many Pareto charts focus on problems.

pa re t o c h a rt s

4. Enter a vertical scale on the right-​hand edge of the chart, running from 0 percent to 100 percent. 5. Draw a line from the bottom left edge of the figure to the top of the first bar. Then extend it to where the second bar would be if the first two bars were stacked on top of one another. Continue this pattern for the third and subsequent bars. This line is a cumulative percentage curve. It reflects the percentage of the total problem comprised by the first bar, and then by the first two bars, the first three bars, and so forth, as shown by the scale on the right-​hand edge of the chart. Continue this pattern for all the bars, eventually including 100 percent of the problem.5 Many Pareto charts focus on the frequency of a particular occurrence—​ for example, complaints generated by a given process or directed toward a particular department, returns for rework at the fleet maintenance garage, checks issued in error by accounts payable, or errors detected in plan reviews in the development review process. Alternatively, the Pareto chart could focus not on the frequency of problems but on the costs imposed by the problems. The first bar in the chart would feature the costliest category of problem, not necessarily the most frequently occurring one. In yet another variation, the Pareto chart can focus on the allocation of worker time to various tasks. By rank-​ordering tasks by the number of work-​hours devoted to each, analysts have focused efficiency improvement efforts toward tasks where the opportunity for significant yield was greatest. The Pareto chart shows where attention should be directed first. If two or three bars constitute more than half of all problems, it makes sense to focus there. “Experience has shown that it is easier to reduce a tall bar by half than to reduce a short bar to zero.”6 Making big gains in the reduction of large, persistent problems would yield the greatest payoff. Whether or not to proceed subsequently to tackle lesser categories becomes a matter of assessing the likelihood of diminishing returns.7 As noted previously the Pareto Principle is a rule of thumb, not a law of nature. When a Pareto chart does not reflect a relationship in accordance 5 Donald J. Wheeler, Making Sense of Data (Knoxville, TN: SPC Press, 2003). 6 Ishikawa, Guide to Quality Control, 45. 7 Wheeler, Making Sense of Data, 49.

303

304

process imp ro v em ent

with the principle—​that is, the chart does not show a few categories dominating all others—​its value as a guide to where improvement efforts should be directed is diminished. A flat Pareto chart—​one that does not contain two or three dominant bars—​is telling managers that the problems are widespread and another tool or a further assessment may be needed in order to target improvement efforts properly. A comparison of Pareto charts prepared before and after an improvement initiative should show a reduction in the major problem that became the primary focus of the initiative. It should also show a drop in the overall total when combining all problem categories. In many cases, a new order will be established among problem categories, with the earlier top category moving to a lower rank. BOX 24.1  PAD-​AND-​PENCIL EXPLORATORY PARETO ANALYSIS In cases where data have not already been collected about various categories of problems or their possible causes, the concepts of Pareto analysis may still lend themselves to a rudimentary exploratory effort. Imagine, for instance, that a fire chief is concerned about slow turnout times for fire crews—​the time it takes in emergencies for firefighters to don their gear, get on the truck, and depart the station—​ but this fire department has not previously collected data on possible causes for slow turnouts. The chief or an assistant could compile a list of possible causes of delay and begin tracking on a simple tally sheet the apparent cause each time a turnout took longer than the standard amount of time. For each such occurrence, someone would make a tally mark against such categories as “late-​night call,” “clutter in the station,” “call during a meal,” or other possible explanations for a slow turnout. Generally, tallying three or four dozen of these problem events can be enough to uncover a range of problems—​perhaps even some not initially anticipated—​and reveal whether an uneven distribution among the causes exists. This, of course, is what the Pareto analyst is looking for. Even this pad-​and-​pencil version of Pareto analysis may be enough to focus the problem-​solving effort on the one or two most important causes and thereby increase the chances of making significant improvements.

pa re t o c h a rt s

Back in Fiddler Sonja Twain collected details of all the complaints received by or about the public works department for the previous 12 months. She eliminated items that were just calls for information even though they were tracked in the same database as the complaints. She sorted the complaints into the top nine categories and put all of the remaining complaints in a category marked “other.” When Sonja constructed a Pareto chart using the 10 categories, she found that the complaint frequencies of the three categories mentioned by Tom McGraw all made it into the top four. Street potholes were the most frequent complaint (Figure 24.1). Complaints about idle workers were a distant third and complaints about the fall cleanup an even more distant fourth place. “I suppose this result shouldn’t surprise me,” Sonja thought, “these three were important enough for citizens to mention them to the public works director, himself.” Complaints about trash in roadways and along rights-​of-​way also made the top four.

Figure 24.1  Pareto Chart of Public Works Complaints in Fiddler

305

306

process imp ro v em ent

After arranging to meet with Tom, Sonja entered his office and laid her chart in front of him. “More than three-​quarters of our complaints fall into three categories, including two of the three types of complaints you mentioned at the outset. This should help narrow our focus as we decide where to start making improvements.” “This is helpful, Sonja,” Tom said. “I look forward to showing the chart to Tyler and getting his reaction. Any reason we shouldn’t begin with the pothole complaints as our focus?” “That seems reasonable to me,” she said. “But I should mention that some authorities on Pareto charts remind us to think not only about the frequency of problems but also their costs. If you think the costs associated with one of the others is greater than for potholes, that might affect your choice of where to start.” “Thanks, Sonja. Good advice,” Tom said. “Because of the wasted resources when crews are idle, I might want to elevate that one to top priority status, even though it’s only third on this chart. I will give this some further thought. Your Pareto chart is really helpful.”

References Konz, Stephan, and Steven Johnson. Work Design: Occupational Ergonomics. 7th ed. Boca Raton, FL: CRC Press, 2016. Ishikawa, Kaoru. Guide to Quality Control. Tokyo, Japan: Asian Productivity Organization, 1982. Wheeler, Donald J. Making Sense of Data. Knoxville, TN: SPC Press, 2003.

Suggested for further information Brassard, Michael and Diane Ritter. Memory Jogger 2. Methuen, MA: Goal/​ QPC, 2018. Joiner Associates. Pareto Charts: Plain & Simple. Madison, WI: Joiner Associates Inc., 1995.

Web resources American Society for Quality, “What Is a Pareto Chart?” https://​asq.org/​ quality-​resources/​pareto

pa re t o c h a rt s

The Minitab Blog, “When to Use a Pareto Chart” https://​blog.minitab.com/​ blog/​understanding-​statistics/​when-​to-​use-​a-​pareto-​chart Minnesota Department of Health, “Pareto Chart” www.health.state.mn.us/​ communities/​practice/​resources/​phqitoolbox/​pareto.html Walsh-​Kelly, Christine M., “Pareto Chart” ​www.med.unc.edu/​pediatrics/​files/​ 2018/​01/​pareto-​chart_​09_​13_​2016.pdf For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

307

25 CAUSE-​AND-​EFFECT DIAGRAMS

When problems emerge in a local government operation—​problems that affect the results that a department or program is achieving—​conscientious managers and supervisors will try to diagnose and fix those problems. The problems may be manifested, sometimes dramatically, in adverse incidents or less dramatically in complaints or declining performance measures. Perhaps the hiring of new employees is taking too long, juvenile crime is growing out-​ of-​hand and crime statistics are on the rise, too many errors are being made by accounts payable, or the requisition process is much too slow and cumbersome. Each of these may flare up in a controversial incident or build over time as a nagging problem. Sometimes the cause of a problem is obvious and the remedy can be swift and complete, but often the cause is less obvious and attempted remedies by an individual manager or supervisor fail to have the intended effect. The true cause of the problem in such instances might have escaped the supervisor’s detection; the cause might be more multifaceted than the supervisor realizes; it might involve more units than the supervisor’s alone; or the underlying cause could be more subtle or complex than the supervisor realizes. An alternative to the Lone Ranger approach to problem solving is one that taps the insights of a group of persons in the development of a cause-​and-​effect diagram focusing on the problem at hand. Rather than relying on problem

c au s e -a n d -e ffe c t d i agr a m s

diagnosis and strategy design by a single manager or supervisor, this approach brings together a group of persons knowledgeable about and engaged in the process under review, and invites them to brainstorm the possible causes of a given problem.1

Scenario: Fiddler, Tennessee, Redux Tom McGraw, the public works director introduced to readers in Chapter 24, knew that his department faced public criticism for potholes and roadway trash, but the category of complaints that worried him most—​and the issue he wanted to tackle first—​was the criticism his department was receiving about idle crews. Complainants reported crew members just standing around at job sites or even sitting and visiting rather than working. Whenever he mentioned these complaints to operations superintendent Tyler Swift, he always got the same response. Tyler always blamed the problem on the lousy work ethic of today’s workers, their general laziness, their need for constant supervision. His solution was to stay on them and push them harder. Now Tom was meeting with his administrative assistant Sonja Twain to decide how to proceed. “Tyler’s solution is to push the crews harder and monitor them more closely, but I suspect that this problem has a broader range of causes than that,” Tom said. “I want a more thorough diagnosis.” “What do you have in mind?” Sonja asked. “Tyler’s perspective is important, but it’s not the only perspective on this problem,” Tom replied. “Let’s assemble a small group of people who understand the work being done by these crews, how the crews are scheduled and deployed, how they are trained, how they are equipped, how they are supervised, how they and their supervisors interact with citizens, and what problems they may encounter in trying to do their work well. This group should not consist solely of managers and supervisors. There should be at least one or two frontline workers in the group, too. Let’s see what this group thinks are the causes when public works crews are idle. The product

1 Because this approach to problem solving draws on a wider range of perspectives, Cohen and Brand suggest that it reduces the possibility of “jumping to ill-​advised solutions.” See Steven Cohen and Ronald Brand, Total Quality Management in Government: A Practical Guide for the Real World (San Francisco, CA: Jossey-​Bass Publishers, 1993), 96.

309

310

process imp ro v em ent

will be a cause-​and-​effect diagram that should help us devise our strategy for making some improvements.” “What will Tyler say about this?” Sonja asked. “Have you spoken to him yet?” “I met with him this morning,” Tom replied. “He’s on board, although I won’t say he’s enthusiastic about it. He questions whether it’s really necessary, but he promises to be helpful. I told him that you would facilitate the process and associated meetings. I hope you’re okay with that.” “Sure, I would be happy to do that,” Sonja promised.

Cause-​a nd-​e ffect or fishbone diagrams Cause-​ and-​ effect diagrams were introduced more than half a century ago by Kaoru Ishikawa of the University of Tokyo in work he was doing with Kawasaki Steel Works.2 The use of these diagrams as quality improvement and control tools spread throughout Japanese industry and soon to other countries, including the United States. These diagrams—​also called Ishikawa diagrams or fishbone diagrams—​have been widely used in quality circles and other efforts designed to identify all the factors that contribute to the success or failure of a given process and to elicit ideas about steps to minimize problems or exploit opportunities. The common fishbone label for cause-​and-​effect diagrams comes from their resemblance to the skeleton of a fish, with the problem to be solved displayed in the “fish head” (Figure 25.1). The problem is the effect that is influenced by the various causes that are then identified in the diagram. Placed in the fish head is a concise statement of the problem. What is the deficiency in the product or service that needs to be remedied? Is it missed garbage collections? Is it failure to adequately prepare athletic fields for team sports? Is it slow response to emergencies? Is it poor work attendance or a poor worker-​safety record? Clearly stating the problem is crucial to making this tool work. A vague statement of the problem almost inevitably leads to a less helpful diagram. For example, a somewhat vague problem statement of “slow fire response times” would be less helpful than a more 2 An account of the introduction of these diagrams is provided in Kaoru Ishikawa, Guide to Quality Control (Tokyo, Japan: Asian Productivity Organization, 1982), 29.

c au s e -a n d -e ffe c t d i agr a m s

Organization

Materials

People

Statement of Problem

Other

Equipment

Methods

Figure 25.1  Basic Cause-​and-​Effect or Fishbone Diagram

precise statement that “37 percent of all fire calls took more than the standard of 5 minutes and 20 seconds for arrival of the first unit.” Whatever the problem is, it should be written in at the end of the cause-​and-​effect diagram, indicating the diagram’s focus. All the branches and twigs—​or bones in the fishbone diagram—​lead to and contribute to this problem. The major bones coming diagonally off a central backbone in the diagram are the categories of causal factors that are thought to influence the effect. Typically, these are major categories—​for example, people, methods, materials, equipment, and organization—​that appear often as the major bones in a fishbone diagram for various problems. Minor bones coming off the major bones more clearly identify the specific causes. Minor bones added to the people bone might be “insufficient skills,” “miscommunication,” “inadequate supervision,” “poor judgment,” or others. Each of these factors constitutes a secondary branch in the diagram. More detailed factors can be added as twigs on these secondary branches. Although the labels noted above are common, the major bones in a cause-​ and-​ effect diagram—​ the major categories of causes—​ may differ from one diagram to another. Ishikawa suggested there are four common reasons for quality dispersion: the raw materials used in production; the tools or equipment used; the work method, process, or workers; and the

311

312

process imp ro v em ent

precision and reliability of measurement or inspection.3 These were usually depicted as major branches in his diagrams. Other authorities have different sets as the major branches in most of their diagrams—​for example, • • • •

human, operating methods, equipment, and working conditions people, machines, measurements, methods, and materials materials, operators, equipment, procedures, and environment people, methods, materials, and organization

Although these or similar labels are found in many cause-​ and-​ effect diagrams as the major branches, other sets of causal factors may be more appropriate for a given diagram. The key to the effectiveness of the diagram is to make it fit the problem being addressed with as complete a set of causal factors as possible. Experience suggests that it is wise to begin the diagram’s construction with a set of major categories under which more specific causes of the problem will be placed. The categories noted in the preceding paragraph are examples. These broad categories will serve as prompts to help the persons developing the diagram resist the temptation to focus solely on the usual suspects rather than considering a full array of possible causes. Typically, the process of developing a cause-​and-​effect diagram begins with a statement of the problem the organization is experiencing and proceeds to a brainstorming session to generate a list of possible causes of a problem. Although major categories of presumed causes—​the “major bones”—​may be entered at the outset to guide the identification of specific causes, an alternate, more freewheeling approach may be taken. The steps in constructing a cause-​and-​effect diagram in this more open-​ended manner are the following: 1. Identify the problem or matter of concern. This is the effect in the cause-​and-​effect diagram and should be written into the diagram at the end of a long horizontal arrow. 2. Make a separate list of all the possible causes of the problem—​that is, the factors that might influence the effect of concern. Do not put them on the diagram just yet. 3 Ishikawa, Guide to Quality Control, 18–​20.

c au s e -a n d -e ffe c t d i agr a m s

3. Organize the causes into groups. Label the groups. These labels will be the major branches off the horizontal arrow in the cause-​and-​effect diagram. 4. Organize the causes within each group, identifying those that are major sub-​branches and those that are minor. Eventually, all causes should be listed on the diagram. 5. Check the diagram for completeness.4 The group is likely to refine its product upon review. Overlooked causes will be added; major sub-​branches will be changed to minor, and vice versa; and labels will be modified to be made more accurate. To be a finished product, the group needs to be satisfied that the diagram identifies all of the cause-​and-​effect relationships relevant to the problem at hand.

BOX 25.1  POST-​IT NOTES AND CAUSE-​AND-​EFFECT DIAGRAMS A tactic that is often helpful to groups working on a cause-​and-​effect diagram calls for the use of Post-​it Notes.1 Once the problem has been defined, each member of the group is asked to list on Post-​it Notes all the causes they can think of that contribute to the problem, one cause per note. Beginning this way increases the likelihood of full participation in identifying causes and ensures that no participant dominates the brainstorming session. Once they have completed this task, each participant attaches their set of causes to the wall. When all the notes have been posted, group members can begin to organize the notes, clustering related causes together. Often, this leads to the identification of major categories for the cause-​and-​effect diagram. This technique for creating cause-​and-​effect diagrams is often more energizing and engaging than the alternative of assembling around a table and brainstorming as a group without first soliciting individual input. 1 This Post-​it Note tactic is recommended in David Straker, Rapid Problem Solving with Post-​it Notes (Aldershot, UK: Da Capo Press, 1997).

4 This list is drawn from Donald J. Wheeler, Making Sense of Data (Knoxville, TN: SPC Press, 2003), 24–​25.

313

314

process imp ro v em ent

Once the cause-​and-​effect diagram has been refined, the next step is to design a strategy for addressing one or more of the causal factors in ways that will yield a better effect. This could be a strategy to focus on a single factor thought to be the dominant reason for the problem, with the intent of making a dramatic change to minimize that factor’s influence; a strategy of making smaller changes in several factors; or some other strategy designed to reduce the problem. The logic behind Pareto charts (see Chapter 24) may be relevant to the selection of the proper strategy. If one or two factors can be identified as predominant causes of the problem, it makes sense to focus the strategy there. Revisiting the diagram after a strategy has been enacted and results have begun to emerge will give managers and their staff an opportunity to refine the diagram based on new insights. It will also provide a good venue for reviewing the effectiveness of initial strategies and considering whether new strategies are needed.

Back in Fiddler Sonja Twain met with Tyler Swift to decide on the composition of the task force that would be charged with developing the cause-​and-​effect diagram. They chose five members: a public works supervisor, three members from different public works crews, and Tyler. Sonja would facilitate the meeting. When the task force gathered in the conference room the next week, Sonja greeted everyone and thanked them for helping out. She explained why they were there and what she and the public works director hoped they could accomplish. “Tom McGraw is concerned about the growing number of complaints from citizens, reporting that they have seen crews just standing around, not working. We are hoping that you can help us understand why this issue keeps coming up and, if it’s true that our crews aren’t working as much as they should be, what the reasons are.” Sonja had already drawn on the board a long horizontal line with an arrow pointing to a box. Inside the box, she had written, “Idle crews.” She pointed to the board and explained, “I want each of you to help us identify all of the possible causes of idle crews. We will develop a list and add every possible reason we can think of. Then we will sort all of the causes into logical groups and begin adding them to our diagram.” She asked if anyone had questions. When no one responded with a question, they began.

c au s e -a n d -e ffe c t d i agr a m s

Tyler spoke first. “I think it’s all about human nature. If a supervisor isn’t on-​site with them, crew members take it easy.” “I don’t think that’s it,” objected one of the other task force members. “There are plenty of other reasons.” Sonja quickly spoke up. “Rather than doing this out loud at first, I’d like you to write down your ideas on the Post-​it Notes it front of you, one idea per note.” (See Box 25.1 for a description of this technique.) “You can use as many slips as you need. When you are ready, please stick your notes up on our board here in the categories that you think they fit best. Then we’ll start working on the diagram as a group.” Sonja already had written on the board five major categories: equipment, people, materials, organizing and scheduling, and other. She knew these labels might be changed in the final diagram. The “other” category reserved a spot for any causes she had not anticipated. The group quickly set to work and began tearing off multiple Post-​it Notes. Sonja was pleased to see that all were writing out more than one or two causes and were eagerly posting their thoughts on the board. When everyone had returned to their seats, Sonja led the group on a quick pass through the notes—​clustering similar thoughts, removing duplicates, and getting explanations when needed. For instance, one note just said, “Sent the wrong crew.” Another said, “Job not ready.” And another said, “Tired.” When Sonja read them to the group, these were the explanations: “Most of the time when we’re standing around it’s because they’ve sent out a four-​person crew for a two-​person job.” “Or we get to the job site and find that stuff that was already supposed to have happened hasn’t happened. Like they were supposed to have marked utility locations or they were supposed to have someone on site to direct traffic. If that stuff hasn’t happened, we end up standing around for a while.” “And sometimes, if we’ve been working hard, we just need to catch our breath for a few minutes. Somebody driving by may not think about that.” By the time an hour had passed, new causes were mentioned, new notes were added to the board, and eventually the conversation began to slow down. As they looked over their list of causes, they decided that they fit comfortably under four major categories but with slightly different labels: people, equipment, materials, and organizing and scheduling (Figure 25.2). The people category included not only the work ethic problem—​ Tyler’s

315

316

process imp ro v em ent

Equipment

People Poor work ethic

Wrong equipment

Oppressive supervisor style

Require supervision

Outdated

Top-down orders

No pressure to hustle

Not appropriate for task

No clear direction

Low pay

Not available at time Poorly maintained Frequent breakdowns

Yelling not explaining

Fatigue

Not motivated

Hard labor

Subpar performance

Hot days Problem statement

Needed work breaks

Not delivered

Complaints about idle workers on job sites

Mismatch of work and resources

Inventory management

Wrong crew size

Site delivery

Wrong crew skills Plan doesn’t match work Wrong materials

Logistical failure Materials not delivered Insufficient amounts Special equipment not rented

Cheap items Poor quality

Prep work not done

Traffic supervision not planned

Markings not made Design plan not provided Removal of landscaping Materials

Organizing and scheduling

Utility locations

Figure 25.2  Cause-​and-​Effect Diagram for Idle Public Works Employees

favorite—​but also supervisory attitudes that can dampen worker morale and commitment. The organizing and scheduling category included the problems of mismatches of jobs and crew sizes and the logistical failures when pre-​arrival preparations have not been completed. When Sonja finished the fishbone diagram, she stepped back and asked the task force members what they thought. “There are more causes than I realized,” one said. “Me, too,” several others replied. “I think we have probably done enough for today,” Sonja said. “I’ll share a polished version of the diagram for you to review. Let’s plan to get together again in a few days. Between now and then, please be thinking about any causes we might have overlooked that should be added to the diagram and also think about things that could be done to reduce or eliminate some of the negative influences we have identified and improve our performance.” As the task force members left the conference room, there was a buzz that trailed them into the hallway. They continued to talk among themselves about the day’s topic. Sonja felt good about the progress they had made. It felt as if they had moved past grousing about the complaints and were now thinking about solutions to address the problem.

c au s e -a n d -e ffe c t d i agr a m s

What went right in this instance? Persons knowledgeable about a given process will have many ideas about the possible causes of a problem that is occurring in that process. A group conversation can quickly generate a lengthy list of possibilities, especially when the group is comprised of persons who view the process from different perspectives. The discipline introduced into the conversation by the task of developing a cause-​and-​effect diagram can minimize the repetition of an idea following its first mention—​something that occurs frequently in a general gripe session. The task force in Fiddler worked well together, perhaps because as facilitator Sonja made sure everyone knew that their contribution was important and would be valued. The resulting cause-​and-​effect diagram drawn with the input of supervisors and frontline workers was probably superior to what would have been drawn by supervisors alone or one drawn by McGraw and Twain without the input of others. In its final form the diagram began to lose some of its sleek fishbone appearance, as some of the branches acquired more sub-​branches than others. But this more complete diagram will serve as a better guide for public works leadership as they decide where to begin in fixing the problem.

References Cohen, Steven, and Ronald Brand. Total Quality Management in Government: A Practical Guide for the Real World. San Francisco, CA: Jossey-​ Bass Publishers, 1993. Ishikawa, Kaoru. Guide to Quality Control. Tokyo, Japan: Asian Productivity Organization, 1982. Straker, David. Rapid Problem Solving with Post-​it Notes. Aldershot, UK: Da Capo Press, 1997. Wheeler, Donald J. Making Sense of Data. Knoxville, TN: SPC Press, 2003.

Suggested for further information Brassard, Michael and Diane Ritter. Memory Jogger 2. Methuen, MA: Goal/​ QPC, 2018. Brue, Greg. Six Sigma for Managers: 24 Lessons to Understand and Apply Six Sigma Principles in Any Organization. New York: McGraw-​Hill, 2005.

317

318

process imp ro v em ent

Inoue, Michael S., and James L. Riggs, “Describe Your System with Cause and Effect Diagrams.” Industrial Engineering 3, no. 4 (1971): 26–​31. Konz, Stephan, and Steven Johnson. Work Design: Occupational Ergonomics. 7th ed. Boca Raton, FL: CRC Press, 2016.

Web resources American Society for Quality, “Fishbone Diagram” https://​asq.org/​quality-​ resources/​fishbone Balanced Scorecard Institute, “Cause-​and-​Effect Diagram” https://​balanced scorecard.org/​wp-​content/​uploads/​pdfs/​c-​ediag.pdf Juran, “The Ultimate Guide to Cause and Effect Diagrams” www.juran.com/​ blog/​the-​ultimate-​guide-​to-​cause-​and-​effect-​diagrams/​ Minnesota Department of Health, “Fishbone Diagram” ​www.health.state. mn.us/​communities/​practice/​resources/​phqitoolbox/​fishbone.html Swan, Elisabeth and Tracy O’Rourke, “How to Use a Fishbone Diagram” https://​goleansixsigma.com/​webinar-​use-​fishbone-​diagram/​

26 PROCESS FLOW CHARTS

The work processes of local government can sometimes be complicated and confusing. It is not that the processes were built to be intentionally complex. Their complexity is more often attributable to the variety of different inputs or circumstances that must be accommodated by the process and the number of different departments, offices, or specialties involved. Sometimes the complexity has evolved over time, as new steps were introduced in the process to overcome one problem or another. Process flow charts—​ also called process maps—​ lay out the pathways through a given process and show the twists and turns along the way. Often these pathways accommodate different characteristics of the input. For instance, just ask a purchasing official at the city of Springfield, Massachusetts, about the process for acquiring needed items and the official is likely to answer, “It depends.” If the item being purchased is valued at more than $100 but less than $5,000, the route to a requisition is straightforward (Figure 26.1). If less than $100, it is even simpler: just make the purchase and submit the invoice. On the other hand, if the value is at least $5,000 but less than $10,000, a contract will be needed first. If the value is even greater than that, either a written quote or a formal bid will be required, depending on how much greater than $10,000 it is. And this only gets the process started. The full process of

320

process imp ro v em ent

Figure 26.1  Flow Chart for the Purchasing and Accounts Payable Process at the City of Springfield, Massachusetts Source: City of Springfield, Office of Internal Audit, Citywide Purchasing and Accounts Payable Performance Audit Highlights (Springfield, MA: Office of Internal Audit, September 2018), p. 4.

p ro c e s s flo w c h a rt s

seeking, securing, and paying for needed goods and services is multifaceted with many turns and many hands involved along the way. A process map or flow chart shows each step in the process. For the astute manager or analyst, the chart can sometimes reveal opportunities for process improvement—​perhaps places where procedures can be streamlined or unnecessary steps eliminated. Some process charts include what are commonly known as swim lanes to show how the process may move across different parts of the organization or through different subprocesses. Figure 26.1 shows these swim lanes for the purchasing process as it moves from the department to the office of management and budget (OMB), then to procurement and the comptroller, before finally ending with the treasurer cutting a payment check. Including these swim lanes can reveal where a process may slow down or get stuck as it moves from one office to another.

Scenario: Procedural problems in Maybree County, North Carolina Department heads and supervisors in Maybree County, North Carolina, have been increasingly frustrated by the time-​consuming procedures required to recruit and hire replacement employees whenever vacancies occur. The county’s chief administrative officer (CAO) Andrea Taylor has parried the complaints of one department head after another with reminders of the  importance of a centralized human resources (HR) department for handling all the posting, advertising, and processing chores, and making sure that Maybree County complies with all federal, state, and local laws and regulations. Today, however, Barney Fike, the county’s public works director, stormed into her office and began explaining how current procedures were hurting his department’s productivity. “I’m not opposed to centralized HR,” Barney said, “but my department goes to a lot of effort to schedule our work in advance and arrange for having the proper crews with the right number of members at the right jobs, all to maximize our productivity. When I have to adjust crew assignments because I don’t have workers who should have been hired weeks ago, we’re not getting full advantage from our work planning and scheduling efforts. It seems to take forever to get a new employee hired. Work that I promised citizens we would have completed by now hasn’t even been started.”

321

322

process imp ro v em ent

After discussing the problem at some length, Andrea decided that it was time to take a closer look at the hiring process. She summoned HR director Floyd Barber to her office and asked him to consult with the public works director and prepare a detailed evaluation of current procedures to see if some improvements could be devised. The two men decided to begin their analysis by developing a process flow chart for the hiring process, showing the steps from the time a vacancy occurred until the job was filled.

Process flow chart Process flow charting is a technique for documenting the steps in a recurring process. Flow charts come in different varieties, some with more detail than others. The flow chart presented earlier for purchasing and accounts payable processes in Springfield showed the different routes for purchases of various amounts all the way through to the eventual payment of vendors. Consider also a flow chart showing the steps involved in removing inoperable or abandoned vehicles from the streets in San Jose, California (Figure 26.2). Removal of vehicles may be initiated by any of three parties: a citizen (customer) calling or using the My San Jose app, the police department (PD), or the department of transportation itself (DOT). The process, which includes a step designed to catch duplicate requests, involves an initial visit to the vehicle in question to be sure that towing is warranted. If a tow is warranted, a tag is placed on the vehicle to alert the owner and provide the owner an opportunity to move the vehicle and avoid a tow. Some charts focus on a small segment of a process, providing granular details showing all the steps taken by individual workers. These charts can be especially helpful for considering ways to improve efficiency or service quality. By recording in sequence and categorizing each step of the process, inefficiencies are often exposed and opportunities for improvement identified. Little specialized knowledge is necessary for rudimentary process flow charting. Other than familiarity with a set of common symbols that help to categorize elements of the process (see Table 26.1), the primary requisite for process flow charting is the ability to conduct careful observation. The successful use of flow charts also requires perceptiveness and thorough consideration of alternate procedures and strategies. The manager or analyst preparing the flow chart records all the steps involved in the current process and categorizes each step by chart symbol.

p ro c e s s flo w c h a rt s

Figure 26.2  Process for Removing Inoperable and Abandoned Vehicles in San Jose, California Notes: (1) Mark/​tag is the process of marking the tires with chalk to easily detect on a second visit whether the vehicle has been moved and placing a bright orange tag on the vehicle to alert the owner that the vehicle is subject to removal. (2) DOT is Department of Transport, PD is Police Department, and GOA is “gone on arrival.” Source: City of San Jose, California, Audit of Vehicle Abatement (San Jose, CA: Office of the City Auditor, Report 18-​04, August 2018), 10.

Table 26.1  Four Common Symbols in Process Flow Charting Symbol

What does it mean?

▭

Ovals customarily mark the beginning and ending points of a process flow chart. Rectangles represent processes. A diamond indicates a decision. Arrows indicate the direction of flow.

⬭

◇ →

323

324

process imp ro v em ent

Often, it is helpful to engage in the effort two or three people who are familiar with the process—​including front line workers—​to be certain that nothing is overlooked. After documenting the current process, it is important to gain a sense of the time required between the completion of one step and the completion of the next step—​perhaps from a review of records. This is the elapsed time and includes not only processing time but also any delays prior to processing. (Think of this non-​processing time as time in the arrows of the chart rather than in one of the rectangles or diamonds.) Next, the steps can be analyzed in an effort to identify unnecessary or duplicative operations and inspections, unreasonable delays, and components that take longer than they should. A perceptive analyst will question the purpose of each step in a given process, the sequencing of steps, the assignment of duties, and the possible existence of cheaper, faster, or more reliable alternatives. Opportunities for simultaneous rather than sequential processing may be explored. The goal is to eliminate duplication, reduce redundant or excessive inspections, reduce rework loops, eliminate choke points (where a single office or worker is the juncture for too many paths in the process) or bottlenecks, take advantage of electronic transmission of forms and communications when possible, reduce delays and unnecessary handoffs, and generally streamline the process. Often, a surprising amount of time is consumed not in actual processing but in the delays between processing steps. Finding ways to reduce this time brings major gains without the quality-​of-​service downside that can come with pushing workers to just work faster. If a new process is needed, a new process flow chart can be prepared showing the proposed procedures, thereby providing a simple means of comparing the present and proposed methods and highlighting the advantages of the latter.

Back in Maybree County: A streamlined approach When Barney and Floyd met in the HR director’s office a few days later, Floyd pulled out an old process flow chart his department had developed at the time the current process was adopted.1 According to that chart, the 1 This scenario in fictional Maybree County is based on the experience and flow charts of the city of Richmond, Virginia. City of Richmond, Human Resources: Citywide Recruitment and Retention, Report #2016-​06 (Richmond, VA: Office of the City Auditor, March 8, 2016).

p ro c e s s flo w c h a rt s

entire process should take 76 days. “Well, I know that’s not happening,” Barney said. “It takes a lot longer than that.” “I am afraid you’re right,” Floyd said. “Our process takes longer than 76 days, but I’m not sure how much longer. I hope we find that we’re not too far off of that.” The two men reviewed the steps displayed in the old flow chart and discovered that a couple of steps either had been overlooked when the chart was prepared or had been added to the process during the intervening years. They added those steps to a new chart and began pulling records of recent recruitments. When they found that the timelines of the first five recruitments they examined far exceeded the projected 76 days, they knew they needed to check more records to get an accurate picture of how long the process actually takes. Compiling this information was time-​consuming, so Floyd offered a suggestion. “You and I can’t get very far on this project this afternoon,” he said. “Why don’t I ask one of our HR analysts to pull records for all the recruitments from the past four years. Then when we get back together in a few days, we will have accurate figures for how long each of these steps actually has been taking.” “That sounds good to me,” Barney replied. “I would like to see if we can trim the time for some of these steps—​or maybe eliminate some steps altogether, if we can.” When Barney and Floyd met again, they had the numbers they needed. Floyd had already penciled onto the chart the average times for various parts of the process. On average, it was taking 46 days to go from a vacancy occurring until approval was granted to begin recruitment, then 10 days until the recruitment began, and then 159 days before the job was filled (Figure 26.3). That totaled 215 days. They agreed this was much too long. “It seems to me that we can find places to trim at each stage,” Barney said, “but I am especially concerned about that last stage. If it takes 159 days from the time we advertise until the time we make the hire, I know we’re losing some good applicants who decide to take someone else’s job offer rather than waiting for ours.” The current process required approvals from four officials—​the deputy CAO over the hiring department, the budget director, the deputy CAO for finance, and CAO Taylor—​before recruitment could even begin. “Isn’t that overdoing it a little?” Barney asked. “Suppose the boss would mind if this just required the approval of the budget office and one deputy CAO?”

325

326

process imp ro v em ent

46 Days

Vacancy Date per RTR

RTR Created

Portfolio DCAO Approval

Budget Approval

DCAO Finance Approval

CAO Approval

10 Days from CAO to Job Created

159 Days

Job Created

Budget Approved

Post job

Applications Received

Position Closed

Candidates Screened

Interviews

Job Offer

Background Check

Job Filled

Figure 26.3  Maybree County’s Current Employee Recruitment Process Requires 215 Days to Fill Vacancies Notes: (1) RTR is the “request to recruit” form. (2) Portfolio DCAO is the deputy chief administrative officer responsible for the hiring department. (3) DCAO Finance is the deputy chief administrative officer responsible for finance. (4) CAO is the county’s chief administrative officer. Source: City of Richmond, Human Resources: Citywide Recruitment and Retention, Report #2016-​06 (Richmond,VA: Office of the City Auditor, March 8, 2016), 9.

“Let’s try it,” Floyd agreed. They studied each subsequent step and found several other places to trim. Their new process flow chart was streamlined to require 128 days from start to finish (Figure 26.4). When Barney and Floyd met with Andrea, the public works director eagerly slid the flow charts in front of her. “Just like I said,” Barney declared, “the current process is taking way too long. But we’ve trimmed it down to something way more reasonable.” “You’ll notice,” Floyd added, “we’ve dropped some of the approvals required at the front end—​such as yours.” “Yes,” Andrea replied, “I noticed that right away. I’m OK with that. Usually, I just check to see that the budget director has signed off before I agree anyway.” “We trimmed several other places, too,” Floyd said. “One of the biggest places is actually a step performed by the hiring departments themselves.

p ro c e s s flo w c h a rt s

128 Days

1. Vacancy Date per RTR

2. RTR Created

3. Budget Approval

4. Portfolio DCAO Approval

5. Job Created and Posted

6. Job Unposted

7. Job Interviews

8. Job Offer

Figure 26.4  Proposed Employee Recruitment Process Will Require 128 Days to Fill Vacancies Notes: (1) RTR is the “request to recruit” form. (2) Portfolio DCAO is the deputy chief administrative officer responsible for the hiring department. Source: City of Richmond, Human Resources: Citywide Recruitment and Retention, Report #2016-​06 (Richmond,VA: Office of the City Auditor, March 8, 2016), 11.

Once we’ve screened the applications in HR and forwarded the certified list to the hiring department, we think they should be able to conduct interviews and make the job offer within 14 days rather than the 79 days they’re taking on average now. We think we can tighten that up.” All parties agreed that the revisions looked promising. As a precaution, however, they decided to initiate the revised procedure as a six-​month pilot project involving only new hires for the public works department. If successful after that period, they would consider recommending adoption of the new process countywide. “Just one more thing,” Andrea added. “These charts served their purpose, so I am generally OK with them. But I have seen process flow charts that had a lot more detail about the various steps and that showed whether things were happening concurrently or strictly sequentially. I would like for you to take a stab at putting this in more detailed form. Doing so might give you some additional ideas for streamlining the process. A more detailed form is also the version I would like to use when we roll this new process out to other department heads—​not only because a detailed flow chart will explain the process better but also because it will be a good model for how I would like to see process flow charts done around here in the future.”

Utility of process flow charts Many local government services are performed through complex systems that accommodate differences in the types of cases being handled and sometimes involve multiple offices, departments, or work units. Explaining how the total system works or how a case flows through the system can be

327

328

process imp ro v em ent

difficult without a process flow chart. Reforming a complex system without the benefit of a flow chart would be especially daunting. Every local government process is a candidate for flow charting. It is a technique that can be especially helpful whenever: • • •

a current process seems confusing, erratic, redundant, or there is disagreement about how the process is supposed to work technology or better coordination could allow sequential steps to be handled simultaneously current processes are plagued by delays

BOX 26.1  ANOTHER APPLICATION OF PROCESS FLOW CHARTS The storm water program in San Diego was facing a serious problem.1 Residents counted on the city to build, manage, and maintain a vast storm water infrastructure to carry away storm runoff and minimize flood risk—​ all while protecting the water quality of San Diego Bay, Mission Bay, and the San Diego River. But the municipality was falling further and further behind in maintaining the system and the infrastructure was deteriorating. The primary cause of storm drain pipe failures was the amount of old corrugated metal pipes (CMP) in San Diego’s storm water system. Almost all of the remaining CMP had already exceeded its expected life of only 35 years. Some was being replaced gradually on a planned basis through the city’s capital improvement program (CIP) and other sections of failing or deteriorating pipe were handled on an emergency basis using a sole-​source contractor or, if not an emergency, either placed on the CIP list for future maintenance or assigned to an in-​house pipe repair crew (Figure 26.5). There were several problems with this system. For one thing, projects on the CIP needs list had a long queue and even after approval and funding they still took 3 to 5 years for completion. Second, most of the projects handled on an emergency basis were not really finished when the work was done; they were repaired just enough to quell the emergency and were then added to the CIP list for a more permanent fix. Third, the in-​house pipe repair crew, although a more efficient option than outside contractors, had limited capacity and therefore could not take on a lot of the work. Relative costs of the three methods of addressing pipe failure and deterioration—​emergency repair via contractor, normal CIP

p ro c e s s flo w c h a rt s

Pipe failure or deterioration

Yes

Emergency failure?

No

$$$

Divert funding from other CIP projects

Project requires design?

No No

Crew has expertise and resources available?

Yes Use sole source contractor

Work order created for in-house pipe repair crew

Added to CIP needs list

Project completed in 1−6 months

Permanent repair?

Yes

$$ No

Obtain funding (competes with other CIP projects for funding)

Use contractor

Yes

$

Obtain funding from general fund

Project completed in 1 month−2 years

Repair complete Project completed in 3−5 years (after initiation/ funding)

Emergency CIP

Non-emergency CIP

In-house repairs

Legend:

$

Illustrates comparative cost of repair/rehabilitation

Illustrates rough proportion of pipe repairs/rehabilitations to be completed through each process

Figure 26.5  The Current Process for Addressing Pipe Failures and Deterioration Note: CIP = capital improvement project Source: City of San Diego, Performance Audit of the Storm Water Division (San Diego, CA: City of San Diego/​Office of the City Auditor, June 2018), 21.

329

330

process imp ro v em ent

work by contractor, and work performed by the in-​house pipe crew—​are represented in the flow chart by three dollar signs for the most expensive route, two for second-​most expensive, and one for the most cost-​efficient option. These were serious problems, but the really big, overarching problem was that, at the current pace, it would take San Diego 95 years to replace all the remaining CMP. A study of this situation produced recommendations for change as reflected in a new process flow chart (Figure 26.6). The changes included

Preventative maintenance based on condition assessment

Pipe failure or deterioration

Yes

Emergency failure?

No

No

$$$

Divert funding from other CIP projects

Project requires design?

No

Crew has expertise and resources available?

No

Yes

Project completed in 1−6 months

Permanent repair?

Yes Work order created for in-house pipe repair crew

Added to CIP needs list

Use sole source contractor

$$

No

Obtain funding (competes with other CIP projects for funding)

Use contractor

Yes

$

Eligible for pipe lining?

$

Yes

Contract out for pipe lining

Project completed in 2−3 days

Obtain funding

Project completed in 1 month−2 years

Repair complete Project completed in 3−5 years (after initiation/ funding) Emergency CIP

Non-emergency CIP

In-house repairs

Pipe lining

Legend:

$

Illustrates comparative cost of repair/rehabilitation

Illustrates rough proportion of pipe repairs/rehabilitations to be completed through each process

Figure 26.6  Proposed Process for Addressing Pipe Failures and Deterioration Note: CIP = capital improvement project Source: City of San Diego, Performance Audit of the Storm Water Division (San Diego, CA: City of San Diego/​Office of the City Auditor, June 2018), 25.

p ro c e s s flo w c h a rt s

expanding the capacity for in-​house maintenance and repair, proactive inspections of the storm water infrastructure, and expanded use of pipe linings to rehabilitate pipe sections where appropriate. Pipe lining was a cost-​effective way to extend by 50 years the useful life of CMP. The amount of work shifted by the new process away from emergency and non-​emergency CIP to in-​house repairs and pipe linings can be seen by comparing the circled symbols at the bottom of the two process flow charts. These symbols represent the amount of work anticipated for each repair option. These changes were expected to bring quicker and more efficient pipe repairs, as well as a steady reduction in the pipe rehabilitation backlog. 1 City of San Diego, Performance Audit of the Storm Water Division (San Diego, CA: City of San Diego/​Office of the City Auditor, June 2018).

References City of Richmond. Human Resources: Citywide Recruitment and Retention. Report #2016-​ 06. Richmond, VA: Office of the City Auditor, March 8, 2016. City of San Diego. Performance Audit of the Storm Water Division. San Diego, CA: City of San Diego/​Office of the City Auditor, June 2018.

Suggested for further information Aft, Lawrence S. Work Measurement and Methods Improvement. New York: John Wiley & Sons, 2000, 33–​104. Brassard, Michael and Diane Ritter. Memory Jogger 2. Methuen, MA: Goal/​ QPC, 2018. Damelio, Robert. The Basics of Process Mapping. New York: Productivity Press, 2011. Haynes, Patricia. “Industrial Engineering Techniques.” In Productivity Improvement Handbook for State and Local Government, edited by George J. Washnis, 204–​236. New York: John Wiley & Sons, 1980. Madison, Dan. Process Mapping, Process Improvement and Process Management: A Practical Guide to Enhancing Work Flow and Information Flow. Chico, CA: Paton Press, 2005.

331

332

process imp ro v em ent

Swanson, Richard A. Analysis for Improving Performance, 2nd ed. San Francisco: Berrett-​Koehler Publishers, 2007. Wheeler, Donald J. Making Sense of Data. Knoxville, TN: SPC Press, 2003.

Web resources American Society for Quality, “What Is Value Stream Mapping (VSM)?” https://​ asq.org/​quality-​resources/​lean/​value-​stream-​mapping BEM, “7 Common Process Mapping Mistakes and How to Avoid Them” www.businessmapping.com/ ​ b log/ ​ 7 - ​ c ommon-​ p rocess-​ m apping​mistakes/​ Colorado Department of Transportation, “Process Mapping (Process Flowcharting Guide)” www.codot.gov/​business/​process-​improvement/​ self-​service/​tools/​process-​mapping Enginess, “Beginner’s Guide to Process Mapping” www.enginess.io/​insights/​ beginners-​guide-​process-​mapping Hessing, Ted, “Process Mapping” https://​sixsigmastudyguide.com/​process​mapping/​ iSixSigma, “Practical Guide to Creating Better Looking Process Maps” ​www.isixsigma.com/​tools-​templates/​process-​mapping/​practical-​guide-​ creating-​better-​looking-​process-​maps/​

27 CONTROL CHARTS

Most managers at various levels of a municipal or county government receive financial and performance reports from time to time. Sometimes they are financial reports that show receipts, expenditures, and various financial ratios. Sometimes they are performance reports that show the quantity, quality, efficiency, and impact of work performed by their department or program, perhaps in comparison to targets or to performance during an earlier period. In most cases, these managers attempt to analyze such reports by examining the change—​often the percentage change—​from one period to the next. This might be the change from one month to another, the change from one quarter to the next, the change from one year to the next, the difference between this month’s performance and that of the same month last year, or the difference between year-​to-​date performance this year and year-​to-​date performance for the same month last year. In the numbers presented in financial reports, performance reports, and in the vast array of numbers produced by all the systems functioning in city and county organizations are both signals and noise. Signals can alert a manager or analyst that something important has happened; in such cases, a change in the system or some external influence has caused an important upward or downward movement in the numbers. Noise is quite different.

334

proce ss im prov em en t

In a stable system, noise is more common than signals. Typically, performance numbers do not steadily improve or steadily decline over long periods of time. Instead, they bounce around the norm—​a little better than the norm in one period and a little worse in the next. This happens despite the best efforts of supervisors to get optimum performance from the workers and system they oversee every day. Movement in these performance measures from a little up one month to a little down the next and back again—​all this is simply noise in the system and is known as common cause variance. Yet, the response of managers upon receiving a new performance report sometimes indicates that these managers fail to recognize the difference between common cause variation, which supervisors can do little about, and actual signals in the data. Although a large change discovered in a performance report—​large either in raw numbers or as a percentage change—​should always get a manager’s attention, doing a little further analysis is often the appropriate next step rather than drawing hasty conclusions and indulging in the knee-​jerk reaction of blaming a supervisor, if the sharp change was in an unfavorable direction, or imposing a new rule. The raw magnitude of change and percentage of change from one period to the next are notoriously unreliable indicators of signals in the data.1 They often fail to distinguish signals from noise, and may even lead managers to focus on aspects of performance not warranting their attention while ignoring other aspects that do.

BOX 27.1  IS A CONTROL CHART REALLY ABOUT CONTROL? Readers sometimes think that the whole function of a control chart is to control the output of a process and ensure consistent quality. While a control chart can, in fact, help achieve consistent quality, it does much more. Fundamentally, the chart serves as the “voice of the process” and tells the manager and analysts what the process, as now designed, can deliver.1 If managers want more from the process, they must be willing to redesign it to make better results predictable. Control charts can provide a useful monitoring tool for tracking performance; they can improve the likelihood of success in fixing problems in the system by helping

1 Donald J. Wheeler, Understanding Variation: The Key to Managing Chaos (Knoxville, TN: SPC Press, 1993).

c o n t ro l c h a rt s

Table 27.1  Decoding Control Chart Terminology Traditional terminology

Better terminology

Control chart In-​control process Out-​of-​control process Control limits Statistical process control

Process behavior chart Predictable process Unpredictable process Natural process limits Continual improvement

target when or where the problems occurred; and they can help determine when efforts to improve performance actually move the dial. Because control charts document a process’s pattern of results—​in a sense, the way a process behaves—​they have also been called process behavior charts. This is one of several distinctions between traditional control chart terminology and what might actually be a more accurate label (see Table 27.1). Still, knowing the traditional terminology is important, because the traditional terms are what will be found in most references and on most Internet sites. 1 Donald J. Wheeler, Understanding Variation: The Key to Managing Chaos (Knoxville, TN: SPC Press, 1993), 43.

Scenario: Rockwood, Michigan Scott Alford is the finance director of the city of Rockwood, Michigan. In addition to keeping an eye on the municipality’s financial reports and bank balances, he also periodically scans the management report tracking his department’s performance when it arrives on his computer each month. He usually finds that a quick glance is sufficient. He just checks to see if the department is falling far short of any of its targets or if there are any big changes from the previous month or from the same month last year. On the rare instances that he spots a big difference in an unfavorable direction, Alford has found that usually he can just mention it in the next departmental staff meeting and this will take care of the problem. The number often will be back in general alignment the next month. This month’s management report (Table 27.2) drew Alford’s attention to two entries that were badly missing the department’s targets: the times

335

336

process imp ro v em ent

Table 27.2  Management Performance Report: Finance Department Current month

Last month

Same month last year

Change from

Target

September

August

September

Prior year

Prior month

Percentage of average daily cash invested Number of days to reconcile bank accounts

98%

100%

100%

99%

Up 1 pt.

7.0

5.2

5.4

6.2

−21%

No change −4%

Workdays to post payroll Average workdays to process purchase orders

7.0 5

8.2 5.6

6.4 5.1

6.9 4.9

19% 14%

28% 10%

Vendor invoices processed Percentage of invoices paid within 30 days

65 70%

62 65%

60 69%

58 71%

7% Down 6 pts.

3% Down 4 pts.

required for payrolls to be posted and purchase orders to be processed. In both cases the function took longer to complete in September than it had in August and longer than the target prescribed. Alford decided to discuss this with a new management analyst on his staff, Randall Robinson. Randall joined the finance department team a few months ago, selected for his skills in streamlining operations that he had honed working in another city government. Alford greeted Robinson as he entered his office. “Good morning, Randall. I want to thank you again for the report you prepared last month on automated meter reading. It was first-​rate.” “No problem,” Randall replied. “I enjoy that kind of assignment.” “I have something else I would like you to look into,” Alford said. “A couple of lines in this month’s management report suggest to me that we might have problems in the time it is taking us to post payrolls and process purchase orders. Comparing this month’s numbers to last month’s numbers shows an unfavorable movement that might be a signal of a problem. I would like for you to do a little analysis and see if you agree.” “Sure. I will take a look.” Walking back to his own office, Robinson thought about how he might approach this analysis. In a couple of assignments with his previous employer, he had developed control charts to detect signals in performance data—​and his boss had just mentioned his concern that this month’s data

c o n t ro l c h a rt s

might be a signal of a problem. By the time he arrived at his office, he was convinced that developing control charts was the way to at least begin this assignment.

Sorting the signals from the noise Every process has some variability in it. Consider the process of our daily commute to work. Even taking the same route and using the same vehicle, one’s commute on some days will take a little more time than on other days.2 Any number or combination of tiny impediments, present one day but absent the other, could influence the result in a small way. These contribute to common cause variation and may usually be considered to be noise in the system. On the other hand, the commuter’s decision to sleep-​ in an extra half hour or have an extra cup of coffee before departing for work could cause her to miss the best window for commuting times. These choices change the system and would be examples of special cause variation. Similarly, a decision by the local government to install a series of traffic signals along the commuter’s normal route could extend the time of the commute. This, too, would be a special cause variation that will yield a signal in the commute-​time data that something in the system has changed. Arriving late to work should be considered a signal to the commuter that an adjustment is needed—​perhaps in returning to an earlier departure for work, making a request that traffic officials better synchronize traffic signals, or considering a new route. The difference between signals and noise is not as easy to detect as one might hope. A number that changes a great deal from one period to the next does not necessarily indicate a signal and one that changes only a small amount does not necessarily indicate the absence of a signal. A tool to remedy this problem was introduced a century ago. Walter Shewhart, known as the father of statistical quality control and credited with distinguishing common cause variation from special cause variation, developed the control chart—​also called process control charts, process behavior charts, and schematic control diagrams. The chart focuses on variation in an operational process but relies on analysis that differs a bit from the analysis of variance

2 Kaoru Ishikawa, Guide to Quality Control (Tokyo, Japan:Asian Productivity Organization, 1982).

337

338

process imp ro v em ent

Figure 27.1  Typical Control Chart for a Process

(ANOVA) technique of social science statistics. The function of a control chart is to alert an operator, analyst, or manager whenever a service or production process deviates from its normal range of performance. The key contribution of Shewhart’s control chart is the chart’s ability to distinguish exceptional variance in a process from common variance—​ that is, separating potential signals in the data from probable noise. A control chart consists of a time series graph of performance data with three lines added to serve as visual references for the data points (Figure 27.1). One visual reference line is a central line reflecting the average value of the full set of data. The other two lines are called control limits—​one is the upper control limit (UCL), which marks the upper bounds of normal observations; and the other is the lower control limit (LCL), which marks the lower bounds of normal observations. These are equidistant on either side of the central line and are calculated from the data. When a process is in control, the data points in the time series will cluster near the central line and fall between the upper and lower limits. When the process is out of control, some data points will fall beyond the limits. Except in very rare instances that will be explained later, points on a control chart that fall within the control limits represent common cause variation. These points fluctuate from day to day and even within a single day for a variety of common reasons. The variations are not too great and it would be difficult to pinpoint the reason for a slight upward or downward tick. Giving credit to “good supervision”

c o n t ro l c h a rt s

for an upward tick that is still within the control limits or placing the blame on “poor supervision” for a downward tick still within the control limits may be unjustified—​because a host of factors other than the quality of supervision can influence common cause variation. Points on a control chart that fall outside the control limits are evidence of special cause variation. Because these points deviate so much from the norm, it is almost certain that a special factor has caused the variation. Two mistakes can be made in the analysis of performance data. First, an analyst can think that a data point is a signal of a change when, in fact, it is only routine variation or noise in the process. And second, the analyst can disregard a signal, thinking it is only routine variation (noise). Shewhart labeled these as mistake 1 and mistake 2: •

•

Mistake 1 occurs when managers overreact to routine variations from the norm—​that is, the generally small upward or downward ticks attributable to common cause variation—​and take actions that cannot really be justified on the basis of presumptions about cause and effect. Mistake 2 occurs when managers underreact to unusual variations from the norm—​typically, larger-​than-​normal and sometimes major upward or downward movements attributable to special cause variation—​and take no action, ignoring the possibility that an adverse special cause could recur and therefore should be addressed.

By considering everything to just be noise in the process, an analyst or manager will never make the first kind of mistake but risks making whopping mistakes of the second kind. On the other hand, by treating every variation as a signal the analyst or manager will avoid the second kind of mistake but will repeatedly make the first kind. The key to effective data analytics is sorting out signals from the noise in order to minimize the likelihood of making one type of mistake without greatly increasing the likelihood of making the other type. If managers direct their corrective actions toward performance that has moved outside the control limits and take a more relaxed posture toward performance that remains within the control limits, they will rarely make either type of mistake. The control limits are not perfect boundaries between common cause and special cause variation, but their guidance will rarely be wrong.

339

340

process imp ro v em ent

Constructing a control chart As noted previously, the control chart methodology differs from ANOVA of social science statistics, even if both focus on variation among data points. ANOVA is often used in the analysis of a set of experimental data but rarely used in the periodic or ongoing analysis of observational data such as that found in a local government’s ongoing performance reports. Control charts particularly shine for observational studies in operational settings. Control charts separate potential signals from probable noise by the use of generic and fixed three-​sigma limits.3 A major benefit of control charts is that they not only alert managers, analysts, and supervisors to the presence of special cause variation but also provide potential leads to help identify when and where special cause variation occurred, which can aid in the effort to control or eliminate this cause of variation in the product or service. The construction of a control chart begins with the assembling of a set of observations (data points) on a select dimension of performance. These observations could be the measured results for each of many separate instances of delivering a given service or producing a given product, the averages of all instances over a specified period (for example, the average results for a day or a month), or the percentages of successes during a specified period. Each of these points (the individual instances, the averages, or the percentages) is entered on the data chart. Next, a pair of calculations will be needed. First, the mean value of all observations will be calculated. This will be considered the midpoint of current performance and will define the chart’s central line. Second, a moving range will be calculated using the absolute value of the differences from the first data point to the second, from the second data point to the third, and so forth across all observations. The moving range is sometimes referred to as the two-​point moving range. The difference between the first observation and the second observation is a two-​point range. Similarly, the difference between the second and third observations is a two-​point range, as are the differences between each of the other pairs in the set. The average of all these two-​point ranges is the two-​point moving range.

3 Donald J. Wheeler, Twenty Things You Need to Know (Knoxville, TN: SPC Press, 2009), 8.

c o n t ro l c h a rt s

By convention, the upper and lower control limits (UCL and LCL) are set at three sigma from average performance of a process that is under control.4 Although the mathematics of precise calculations can get complicated, a simplified formula yields control limits that accommodate bias corrections. The resulting control chart is called an average moving range chart or, more commonly, an XmR chart. Below are the simplified formulas: Midpoint of acceptable performance—​ use the average performance calculated from the full set of data Upper control limit = average performance + (2.66 × average moving range) Lower control limit = average performance − (2.66 × average moving range) The average two-​point moving range will be multiplied by 2.66 and added or subtracted from average performance (i.e., the average data point) to find the UCL and LCL, respectively. Having at least 20 to 30 data points is typically desired, but these formulas can be used with as few as five or six data points.5 Typically, a second chart is also constructed, this one plotting the moving range. Although the general procedure for developing this chart is similar to the procedure for the XmR chart, there are some important differences. First, there is no lower control limit. The graph’s floor value of zero serves that function. Second, for calculating the upper control limit of the moving range chart, the multiplier of 2.66 is replaced by a multiplier of 3.27 for the moving range (mR) chart.6 Below is the simplified formula for the mR chart: Central line for the mR chart—​use the average of the moving ranges Upper control limit = 3.27 × average moving range

A word of caution about other approaches to setting the UCL and LCL Some managers have noticed the general characteristics of control charts and attempted to create their own simple charts using methods that differ from those prescribed above. Some, for instance, have just used the standard 4 Wheeler, Twenty Things You Need to Know, 21–​26. 5 Wheeler, Understanding Variation. 6 Wheeler, Understanding Variation, 60.

341

342

proce ss im prov em en t

deviation (σ) statistic (see Chapter 2 “Central tendency and dispersion”) and set their control limits at 1, 2, 2.5, or 3 standard deviations from the mean observation. Others have simply eyeballed the data and drawn in their control limits at places that enclosed most of the observations. And still others have set their control limits to contain the range of observations they would like their process to yield. What is wrong with these alternate approaches? Regarding the use of the standard deviation statistic, this approach confronts a serious methodological issue. Standard deviation is a global measure of variation that assumes a homogeneous data set. In a work setting such as a local government, a stable process that has experienced neither overt nor subtle changes should produce a homogeneous data set, but one that has experienced even subtle changes—​just the sort of changes we might hope to detect in the analysis—​the data set would not be homogeneous and the calculated standard deviation would be an inflated estimate of variation. In contrast, sigma, which is a local measure of variation between sequential points, would produce an estimate of variation similar to the standard deviation if the process experienced no changes and an uninflated estimate if it did experience change.7 Regarding the other two approaches, perhaps W. Edwards Deming said it best when he referred to the setting of control limits arbitrarily as simply “stargazing.”8

BOX 27.2  WHY SET THE UPPER AND LOWER CONTROL LIMITS AT THREE SIGMA? Why set UCL and LCL at three sigma from the central line of the control chart rather than some other distance? Because three-​sigma limits have been found to filter out virtually all routine variation and expose potential signals of exceptional variation.1 The three-​sigma limit is a conservative limit that in practice has been found to balance out the desire to avoid missing a signal, on the one hand, against the desire to avoid overreacting and chasing false signals, on the other hand. Choosing a

7 Wheeler, Twenty Things You Need to Know, 27. 8 W. Edwards Deming, The New Economics for Industry, Government, Education, 2nd ed. (Cambridge, MA: The MIT Press, 1994), 210.

c o n t ro l c h a rt s

two-​sigma limit would increase the rate of false alarms, alerting managers to presumed signals (i.e., special variation) when it was actually just noise (i.e., common variation). Wheeler warns analysts against modifying standard control limits, writing, “Three-​sigma limits have a fine ancestry of high-​brow statistical theorems, and they have also been thoroughly proven in practice. Do not try to reinvent the wheel. Accept no substitutes…. So if a stranger, or your software, or even a friend suggests using something other than correctly computed three-​sigma limits, just say no!”2 1 Donald J. Wheeler, Twenty Things You Need to Know (Knoxville, TN: SPC Press, 2009), 43; W.A. Shewhart, Economic Control of Quality of Manufactured Product (New York: D. Van Nostrand Company, 1931). 2 Wheeler, Twenty Things You Need to Know, 46.

Back at the City of Rockwood Randall Robinson began developing control charts for the two performance measures flagged by the finance director, and decided to add two other entries to his analysis. He decided also to look into “vendor invoices processed” and “percentage of invoices paid within 30 days,” despite the fact that they barely fell short of their targets in September and recorded only small differences from previous months (see Table 27.2). He added them because of the importance of these dimensions of performance for good vendor relations, which keeps potential bidders competing for city contracts. Randall was now focusing on four lines of the management report. Randall had barely begun his task when Russ Adkin, whose office was next door, wandered in for one of his frequent visits. After briefing Russ on his new assignment, Randall showed him the management report for September and pointed to the payroll and purchase order lines. “Those are some bad numbers,” Russ said. “Look at how big the change is from August to September! That’s a bad trend.” “I am going to check to see how much variation is normal to find out whether differences that big are abnormal or not,” Randall replied. “If these numbers are within the range of common cause variation, then it might

343

344

process imp ro v em ent

just be noise in the system and not a real signal at all. It’s unwise to consider an upward or downward move between only two points to be a trend. It takes more data points than that. I will let you know what I find, when I develop this further.” When Russ headed out, Randall got to work on the control charts, beginning with payrolls posted. The numbers reported in the management report looked bad: the 8.2 days required to post payroll in September exceeded the target by 1.2 workdays, exceeded the time in August by 1.8 workdays, and exceeded September of last year by 1.3 workdays. The control chart would either confirm or refute the seriousness of the negative impression made by these numbers and the need for corrective action. Randall’s first step was to gather the data for the past couple of years on how long it took to post the payroll (Table 27.3). He then plotted the data. The time series for the time taken to post payroll for the prior two years and most of the current year is shown in Figure 27.2. Randall used the first two years to establish the baseline for his control chart, which would allow him to see if anything peculiar was happening this year. The average of the 24 values from the first and second years is 7.39 days. This average was used as the central line in the time-​series graph. A glance at Randall’s time-​series figure revealed several spikes—​including an especially sharp one in February 2021—​but otherwise no systematic

Figure 27.2  Line Chart of Time Required to Post Payroll by Month Note: The centerline on the chart reflects the average payroll processing time of 7.39 days during 2019 and 2020.

c o n t ro l c h a rt s

Table 27.3  Days to Post Payroll by Month Year

Month

Days to process

Moving range

2019

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

7.4 7.0 7.8 6.8 6.9 7.1 7.6 7.1 7.3 7.9 7.8 7.6

0.4 0.8 1.0 0.1 0.2 0.5 0.5 0.2 0.6 0.1 0.2

2020

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

8.1 7.9 7.2 7.8 7.9 7.1 7.0 7.9 6.9 7.6 6.8 6.9

0.5 0.2 0.7 0.6 0.1 0.8 0.1 0.9 1.0 0.7 0.8 0.1

2021

Jan Feb Mar Apr May Jun Jul Aug Sep

7.2 9.1 6.8 7.9 7.1 6.9 6.9 6.4 8.2

0.3 1.9 2.3 1.1 0.8 0.2 0.0 0.5 1.8

Note: The moving range is simply the absolute value of the difference between the current point and its predecessor. The average moving range for January 2019 through December 2020 is 0.483 days.

pattern in the data. The data point for September 2021 was on the high side, but the question of whether the September value was an exception remained unanswered. In order to filter out the routine month-​to-​month variation in the data, Randall would need to find the upper and lower control limits (UCL and LCL). To do that, he would first need to measure the

345

346

process imp ro v em ent

month-​to-​month variation. He did this through a two-​step process that yielded the moving ranges: Step one: Randall recorded the differences between each pair of data points in Table 27.3—​that is, the difference between January and February of year 1, the difference between February and March, the difference between March and April, and so forth. The direction of movement is irrelevant; so even if the movement is downward, a positive number is recorded. These are the moving ranges (mR), as shown in the final column of Table 27.3. Step two: Randall added all of the moving ranges together and divided by the total number of paired values to get the average moving range. In this case, it was 0.482 days. He would need this number for the calculation of the upper and lower control limits of his chart. By plugging the average moving range into the formulas, Randall calculated the UCL to be 8.68 days and the LCL to be 6.11 days (Figure 27.3). The control chart showed the September performance to be within the control limits but revealed a signal back in February. Something serious happened back then that should have been investigated. Randall constructed similar charts for the average workdays to process purchase orders and for the two performance measures he added to the

Figure 27.3  Control Chart for Posting of Payroll Data Note: Limits and average based on the first 24 months of data in this example (2019 and 2020).

c o n t ro l c h a rt s

analysis: “vendor invoices processed” and “percentage of invoices paid within 30 days.” As he began reviewing his charts, Randall muttered to himself, “The boss is going to be very interested in this.”

Interpreting the control chart results The UCL and LCL, when calculated as prescribed from the data, are regarded as the natural process limits and serve, in the words of control chart authorities, as the “voice of the process.”9 In other words, these limits are the current process’s declaration of what it can deliver as now designed or practiced. This is what Deming meant when he quoted Irving Burr: “The control chart is the process talking to us.”10 Given a stable system, control limits show what a process will produce tomorrow and in the days that follow. A process that is in control will yield data points that fluctuate around the central line and remain between the control lines.11 Some points in a process that is in control will indicate more favorable performance within normal variation and some will indicate less favorable performance. Some aggressive managers and supervisors may attempt to drive their department or unit to consistently perform near the favorable end of the natural process limits and might even demand from subordinates reasons for failure to deliver that level of performance. However, when the performance lies between the control limits in the territory of common cause variation, looking for explanations as to why some of the data points are higher and some lower than others is regarded by many authorities to simply be a “waste of time.”12 When it is all really just a matter of common cause variation, seeking reasons for good performance marks can lead to false claims of credit and demanding reasons for poor marks can lead to defensiveness and grasping for excuses. Neither is healthy for the organization. Some of the data points in a control chart may leap beyond the control limits and represent signals that something unanticipated is affecting 9 Wheeler, Understanding Variation, 43. 10 Deming, The New Economics for Industry, Government, Education, 178; Irving Burr, Engineering Statistics and Quality Control (New York: McGraw-​Hill, 1953). 11 On the other hand, a process that is out of statistical control will yield patterns of production variation that differ erratically from day to day. 12 Wheeler, Understanding Variation, 25.

347

348

process imp ro v em ent

the system and deserves to be investigated. These sharp deviations from the control limits are the most obvious occurrences calling for management investigation and possible action, but other unusual instances may qualify as signals, too. For example, some authorities have suggested that eight successive values that all fall on the same side of the central line of a process chart warrant careful examination to detect the cause, even if the points remain within the control limits.13 Usually, data points will bounce from one side of the central line to the other, only staying on one side for a few observations before bouncing to the other side, so it is safe to interpret eight in a row as a signal even though no observation leaves the control limits. Still another pattern that could be a signal is when at least three out of four consecutive data points are closer to one of the limits than to the central line.14

Interpreting Rockwood’s charts Randall Robinson found only one signal in the chart for workdays to post payrolls and none in the chart for workdays to process purchase orders. He also found a signal in the chart for vendor invoices paid within 30 days but not in the one for the number of invoices processed monthly. Before reporting back to Scott Alford, he investigated the February signal in payroll and did a very preliminary review of the situation with vendor invoice payments. Then he requested a meeting with Scott. Randall and Scott sat at the small conference table in Scott’s office. Randall placed the “payrolls posted” chart in front of Scott and described his analysis and findings (see Figure 27.3). “For the most part, this has been a stable process. The variation from month to month has been fairly great over the full period of time I examined. Last month’s figure was a little high, but it remained within the control limits. It’s just common cause variation, as far as I can tell,” Randall said. “I think it would be a mistake to make too big a deal of it.” “But your chart shows that the September result was on the high side, approaching the upper control limit,” Scott protested, “and back in February we exceeded the upper limit.” 13 Wheeler, Twenty Things You Need to Know, 86–​88. 14 Wheeler, Understanding Variation; Donald J. Wheeler, Making Sense of Data (Knoxville, TN: SPC Press, 2003).

c o n t ro l c h a rt s

“I investigated the February result, because by veering beyond the control limits it suggested special cause variation—​something different that month must have affected the process,” Randall said. “I found that the clerk who normally handles payroll posting was out of the country on a long-​planned trip and her backup for the payroll posting duty was in the hospital for an emergency appendectomy. We’re talking really special cause variation!” “Okay, not much we can do about that,” Scott said, “but I would rather see a better result on the time it takes to post those payrolls—​something down near the central line or below.” “Sure, I get that,” Randall replied. “But this is what the experts on control charts are talking about when they say the charts are ‘the voice of the process.’ The process is telling us that, as currently designed, we are going to get results between the upper and lower control limits. So for any given month, we might expect the time to complete payroll could range between 6.11 days and 8.68 days and should not be flagged as a problem unless other signals of special variation turn up. We shouldn’t be surprised if the results from month to month are about as often near the upper limit as near the lower limit.” “Well, what if we set our target near the lower limit?” Scott suggested. “Or what if we adopt an incentive system that rewards performance near the lower limit?” “The target is already below the central line,” Randall replied. “Even that might be unrealistic without making some changes to speed up the process. And we ought to think carefully about the potentially negative ramifications of a new incentive system. If a lot of this variation is really not within the control of the operators or their supervisors, it could be a mistake to treat them as if they are causing the good results or the bad results. If we are dissatisfied with any results that fall between the central line and the upper control limit, I think we should commit to a careful review of our current process with the goal of streamlining the process and pushing the central line and both control limits downward.”15 “That sounds good to me,” Scott said. “When can you get started?” “Whenever you say,” Randall replied, “but we might have a greater priority. Let me show you a few other charts.” He placed the 15 See Chapter 26 “Process flow charts” for ideas on process mapping.

349

350

process imp ro v em ent

average-​workdays-​to-​process-​purchase-​orders chart on the table and showed Scott that all the data points were between the upper and lower control limits. “No signal is apparent there, but you’ll be more interested in one of these,” he said, as he placed the vendor invoice charts in front of Scott. “I did these because I know how important it is that we handle vendor relations well.” Randall showed Scott that in both of the vendor invoice control charts—​ the number of invoices processed monthly and the percentage paid within 30 days—​all of the observations remained within the control limits. But then he pointed to a pattern appearing in the most recent months of the chart for invoices paid within 30 days (Figure 27.4). “A year ago we were often beating the target of paying at least 75 percent of our invoices within 30 days, but in the past year we’ve struggled to do so. Look at the data points for the last several months. We know it’s a signal when an observation leaves the control limits,” Randall said, “but it’s also considered a signal if eight consecutive observations are all on one side of the central line, which is what we have here. Something is going wrong with the invoice payment process. The chart shows that the process has been impacted or changed in some manner, resulting in fewer payments within 30 days—​ and it probably started as early as December of last year. The change did not

Figure 27.4  Control Chart for Percentage of Invoices Paid Within 30 Days Notes: (1) Limits and average based on first 24 months of data in this example, 2019 and 2020. (2) The encircled portion of this chart highlights a signal of special variation: at least eight consecutive points falling on the same side of the centerline.

c o n t ro l c ha r ts

produce a dramatic spike like we saw in the other control chart but the dip in performance stretches across 10 months. It appears that something about our process changed even though we didn’t notice it. We should look into it before we start hearing complaints.” “I’m surprised!” Scott exclaimed. “I wouldn’t have guessed we had a problem there.The numbers in the management report for vendor payments just didn’t jump out at me.” “Yes,” Randall replied, “it’s not just about the magnitude of change in the numbers. It’s also about whether the change exceeds normal variation or exhibits an obvious non-​random pattern of variation around the central line. Control charts help us see patterns and major variations more clearly. I did a little exploring of the vendor payment issues and I think we’re going to find that part of the problems are in our processes in the finance department and part are in delays in forwarding invoices from other departments to finance.” “Let’s revise our work plan based on this analysis,” Scott said. “I think you should start with a study of vendor payments. We can hold off for a while on the study of the payroll posting process.”

BOX 27.3  A FEW TIPS WHEN WORKING WITH CONTROL CHARTS An analyst developing or using control charts would be wise to consider these tips: •

•

Decide what measure you wish to use and stick consistently with that measure. You may wish to have each point on the graph represent an observation for each individual worker, the average for a work unit, or the average of all observations for all work units for a day. Measuring any one of these would be fine. But you should not mix different measures in a single control chart. If you do, the analysis will be misleading. Sometimes it might be important to convert raw numbers to rates or percentages for analysis purposes.1 For example, the amount of variation in the raw number of invoices processed or inspections completed from month to month might limit the usefulness of a control chart, but the monthly pattern of variation in “late payments

351

352

pr ocess impr ov em ent

•

•

per 100 invoices” or “percentage of inspections taking more than two days” could be very revealing. If all the data points in a control chart fall between the upper control limit and the lower control limit, does this mean that the process is stable and performance is okay? Perhaps, but it also could mean that the level of data aggregation is hiding some signals. Consider disaggregating the data. For instance, try constructing and examining the control charts of individual crews or workers rather than the performance of the department’s workforce altogether. Breaking out quarterly data into monthly, weekly or even daily data may uncover variation that is concealed when you aggregate upwards over time. It is important to detect and investigate signals in the data, but additional steps must be taken to yield meaningful improvements in the process. “Knowing the assignable cause is only the first step. Detrimental assignable causes need to be eliminated. Beneficial ones need to be made part of the process.”2

1 Donald J. Wheeler, Understanding Variation: The Key to Managing Chaos (Knoxville, TN: SPC Press, 1993), 62. 2 Wheeler, Understanding Variation, 119.

What went right in Rockwood’s handling of this case? What could have gone wrong? It is easy to overreact when a performance measure makes a big move in an unfavorable direction over a short period of time. A manager might think it is a signal when actually it is not. It is also easy to underreact to a small move in an unfavorable direction, thinking it is not a signal when actually it is. Rockwood’s finance director almost made both of these mistakes—​committing mistake 1 by overreacting to the performance information for payrolls posted and purchase orders processed, and mistake 2 by underreacting to the performance information for the percentage of invoices paid within 30 days. But Scott Alford did not make those mistakes. He could have overreacted to the one-​month uptick in time for posting payrolls and processing purchase orders by accusing subordinates of poor supervisory practices or frontline workers of poor effort, by insisting that supervisors

c o n t ro l c h a rt s

provide an explanation for these performance swings, by establishing more ambitious targets and demanding that the workforce somehow meet them, or by making unnecessary adjustments to procedures, all in response to an unfortunate and unexplainable blip within the normal bounds of common-​ source variation. Instead, Alford asked his management analyst to look into the performance changes reflected in the management report and report back with his findings. Luckily, Randall Robinson was familiar with control charts as a means of distinguishing signals from the noise in data patterns. When a manager misinterprets noise as a signal, the subsequent search for an explanation wastes the time and energy of everyone involved in the futile quest for an answer and risks perverse actions taken to avoid blame for a result that was actually only a matter of normal variation. Noise is noise. Nothing more. A control chart filters out the noise. Because Randall also took the initiative to examine the vendor invoice payment metrics and discovered signals in those data, he saved Scott from mistake 2—​failing to recognize and act on signals when they are present. With the benefit of Randall’s analysis, Scott devised a reasonable plan of action. Often, managers interpret changes in performance metrics hastily and incorrectly. “Unfortunately, many who are numerically naïve do not recognize their naïveté. They think that because they can do arithmetic they are qualified to interpret data. Their approach can be summarized as: ‘Two numbers that are not the same are different!’ This theorem of the unconsciously naïve will turn everything into a signal, two points will always define a trend, and explanations will be required for all of the (unfavorable) noise in the monthly report.”16 A manager’s impulse to want better performance than the current process is delivering is not an unhealthy impulse for their organization. This is an impulse that is nurtured among managers, supervisors, and frontline workers in organizations committed to continuous improvement. But if the system is stable—​that is, the system month after month is delivering performance results consistently within the UCL–​LCL range, setting targets outside this range without first changing the process would be an irrational act. For a process under a reasonable degree of statistical control, a better result is likely to be achieved on a consistent basis only if the process is redesigned to make that better result predictable. 16 Wheeler, Twenty Things You Need to Know, 2.

353

354

process imp ro v em ent

Back in Rockwood The next time Russ Adkin wandered in from his office next door, Randall shared with him his findings regarding the processing of payrolls, purchase orders, and invoices, and his method of analysis. Never one to admit he might have been wrong or to acknowledge that he had something to learn as an analyst, Russ only said, “Yeah, I figured it might be something like that.”

References Burr, Irving. Engineering Statistics and Quality Control. McGraw-​Hill, 1953. Deming, W. Edwards. The New Economics for Industry, Government, Education, 2nd ed. Cambridge, MA: The MIT Press, 1994. Ishikawa, Kaoru. Guide to Quality Control. Tokyo, Japan: Asian Productivity Organization, 1982. Shewhart, W.A. Economic Control of Quality of Manufactured Product. New York: D. Van Nostrand Company, 1931. Wheeler, Donald J. Making Sense of Data. Knoxville, TN: SPC Press, 2003. Wheeler, Donald J. Twenty Things You Need to Know. Knoxville, TN: SPC Press, 2009. Wheeler, Donald J. Understanding Variation: The Key to Managing Chaos. Knoxville, TN: SPC Press, 1993.

Suggested for further information Brassard, Michael and Diane Ritter. Memory Jogger 2. Methuen, MA: Goal/​ QPC, 2018. Konz, Stephan, and Steven Johnson. Work Design: Occupational Ergonomics. 7th ed. Boca Raton, FL: CRC Press, 2016.

Web resources Minnesota Department of Health, “Control Chart” www.health.state.mn.us/​ communities/​practice/​resources/​phqitoolbox/​controlchart.html SPC Press, articles by Donald J. Wheeler www.spcpress.com/​reading_​room. php

c o n t ro l c h a rt s

Also, consider a web search using these key words: control charts, process behavior charts, common cause versus special cause variation, and understanding variation. For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

355

Part VI OTHER ANALYTIC OPPORTUNITIES AND TECHNIQUES

Modern local governments—​whetvher small or large—​need to be good at the mechanics of service delivery, but they must be more than that. They also need to be in touch with the needs, desires, and opinions of their citizens and they must manage their resources and services wisely and equitably. The chapters of Part VI can help managers fulfill these responsibilities. Chapters 28 and 29 address the important topics of citizen and customer surveys, focus groups, and the analysis of survey data. Chapter 30 focuses on financial condition analysis and introduces a practical assessment tool that minimizes the need for special data collection by relying instead on data typically reported in a government’s annual financial statement. Chapter 32 addresses forecasting, a topic relevant not only to a local government’s finances but also to program planning in other service departments. Equity is a hot issue for many governments. And appropriately so. Allegations of bias in employment, service delivery, contract awards, or the treatment of citizens and visitors are serious charges that, if substantiated, not only impugn the integrity and professionalism of current officials but

358

other a naly t ic options

also damage a community’s reputation. Chapter 31 introduces binomial probability analysis as a technique for statistically testing one form of bias allegations. It tests the likelihood that the proportion of women, African-​ Americans or Latinos in a government’s workforce or serving on citizen boards could reasonably be attributed to chance rather than to bias in employment and appointment practices. Chapter 33 presents benchmarking as a tool not only for gauging the performance of local government departments relative to their peers but also, in the case of best practice benchmarking, for prescribing performance improvements.

28 CITIZEN SURVEYS AND CUSTOMER SURVEYS

Sometimes the best way to find out how people feel about a particular service or issue is the most direct way—​just ask them. If the service is received by a relatively small number of persons—​say, one hundred or so—​and it is easy to reach them, ask them all. If the number is large or if the questions are directed to others beyond the recipients of a single service, then asking a representative sample often is a more practical way of gauging sentiment.1

Scenario: Idlewild, Delaware Simon Crowell has been in charge of the city’s after-​school recreation program for a year and a half, during which time he has seen participation rates barely move. “I just don’t get it,” Simon muttered, staring at 1 Even if the number of service recipients is not particularly large, local officials might still choose to draw a sample from the community’s population as a whole—​either because they have no registry identifying service recipients or because they also want to hear from nonrecipients to learn why they choose not to receive the service.

360

other a nalytic options

this month’s statistics. “The improvements we’ve made during the last year should have increased participation a lot! The kids in this town just don’t appreciate anything!” “Whoa, Simon! Don’t you think you’re being a little judgmental?” Paula O’Toole is the parks and recreation director who hired Simon and gave him this assignment. She occasionally offers encouragement and direction. “Sorry, Paula, but I guess I’m stumped. I think the classes and other activities we’ve got in place now are really good. The kids should be turning out in bigger numbers,” Simon said. “I don’t know where we are missing the mark.” “Maybe you need better feedback.” “We get evaluation forms from most of the kids whenever we complete a program or activity,” Simon responded. “And most of the comments are positive.” “You’re surveying your customers,” Paula said. “Don’t you need to be hearing from people who aren’t coming if you’re trying to find out why?” “I suppose you’re right about that.” “Why don’t you take a hard look at various surveying options to make sure we are going about this correctly,” Paula continued, “and check out other options, too. You should touch base with Andy Jackson down at the Senior Center. He’s made some programming changes recently that have been very popular. See if he has some ideas that might help.”

Surveys Conducting a survey sounds easy enough, but getting it right is more complicated than it may seem at first blush. Proper randomization (see Chapter 5) is one issue. The sample is supposed to represent accurately the target population as a whole.2 Biased selection of the sample—​even inadvertent bias—​would damage representativeness. Sample size is another issue. Large samples are better than small ones, but sample size quickly reaches a point of diminishing returns (see Table 5.2 in Chapter 5).3 Once basic sample size needs are met, adding more and more 2 In some cases, the surveyors may devise sampling strategies to intentionally overrepresent some small segments of the population in order to have enough respondents from these segments to be able to perform the statistical analyses they have in mind. 3 Mathematical computations based on the desired confidence levels for the survey can be used in lieu of Table 5.2 to prescribe a recommended sample size. For an online sample size calculator, see www.raosoft.com/​samplesize.html.

citiz en s ur v ey s an d c u s t o m e r su rv e y s

respondents brings less and less additional value to the survey. Relatively few community-​wide surveys, even in large cities, include more than 700 respondents. In fact, a survey of approximately 400 to 500 persons, if drawn and conducted properly, will suffice in most cases. Questionnaire design and survey mode are also important issues. Topics addressed in the questionnaire should be suitable for the survey format and appropriate for the array of people being questioned. Asking general citizens if they are satisfied with their water services or if they have noticed any odor lately in their tap water would be appropriate; asking them if they consider the city’s water-​testing program to be adequate or its infrastructure-​financing strategy to be wise would not be. It would be reasonable to assume knowledge about the former, but less reasonable to expect insights among the general populace regarding the latter. Furthermore, questions must be worded and posed appropriately, so as not to influence the respondents’ answers.

AVOID THIS COMMON ERROR BOX 28.1  ASSUMING THAT A RESPONSE RATE OF 20 PERCENT IS GOOD ENOUGH Local governments sometimes include questionnaires with utility bills and achieve a survey response rate of only 20 or 25 percent, which they assume is “good enough” to report the results as representative of the community. Survey experts place the threshold much higher, suggesting that a response rate of at least 50 percent is not only necessary for reliable results but also possible with a good survey plan and reasonable follow up.1 Unfortunately, some low-​cost methods of survey administration are also low-​return, low-​value methods. 1 Harry P. Hatry, John E. Marcotte, Therese van Houten, and Carol H. Weiss, Customer Surveys for Agency Managers: What Managers Need to Know (Washington, DC: Urban Institute Press, 1998), 28; Thomas I. Miller and Michelle Miller Kobayashi, Citizen Surveys: How to Do Them, How to Use Them, What They Mean, 2nd ed. (Washington, DC: ICMA, 2000), 54.

Survey modes continue to expand. Added to the traditional trio of survey modes—​mail questionnaires, face-​to-​face interviews, and telephone interviews—​are newer techniques using email and the Internet. Each mode has its advantages and disadvantages.

361

362

other a naly t ic options

Mail questionnaires are inexpensive, but they often yield poor response rates that can jeopardize representativeness and confidence that the results have value. Face-​to-​face interviews are expensive to conduct, but they usually yield better response rates and allow interviewers to probe responses in greater depth. Telephone interviews have been widely accepted as a generally suitable compromise, providing better response rates than many mail surveys at a lower cost than face-​to-​face interviews. But issues about cell phones and landlines, call screening, and the representativeness of the contacts callers are able to make are difficult to solve satisfactorily. Given the relentless assault of telemarketers and pollsters on every community’s residents, some local governments are reluctant to engage in an activity that contributes to the assault. Online surveys, conducted either by email or the Internet, provide a low-​cost survey option. As with other survey modes, the careful development of a set of survey questions requires time and expense, but, after that, the costs of survey distribution and collection are reduced dramatically. Electronic survey responses often can be fed directly into databases for analysis without requiring a separate round of coding and keying. If citizen or customer email addresses are readily available, electronic surveys can also have targeting advantages. The greatest concerns for this survey mode are randomization across the community population (requiring equal likelihood of being surveyed) and the willingness of contacts to respond. If access is affected by rural–​urban differences, household income, and availability of electronic communication, the representativeness of survey responses will be jeopardized. Local governments that desire the valuable feedback surveys can provide would be wise to secure expert assistance or consult some of the excellent references published on this topic, some of which are listed at the end of this chapter. Careful attention to details and to the rules and nuances of sample selection, questionnaire design, and survey administration can be the difference between survey results that are meaningful and those that are not. Only a few of the many important considerations will be addressed here in brief.

Survey everyone or just service users? Local governments often embark on a general citizen survey or a customer survey without carefully considering the pros and cons of their choice.

citiz en s u r v ey s a nd c u s to m e r s ur v ey s

A general survey of citizens seeks feedback reflecting community sentiment, information about citizen awareness and use of facilities and services, and views regarding the quality or adequacy of local government programs. In contrast, a customer survey seeks the opinions of persons who use a particular facility or receive particular services. Conducting a customer survey allows a department to ask detailed questions about the quality and timeliness of services and get responses from persons who have actually received these services. Depending on the size of the customer base, it might be possible to survey all customers rather than only a sample. Surveying all customers of fire services during the past year—​that is, those who have experienced a fire incident requiring a response from the fire department—​rather than a sample of all the community’s residents would likely produce better informed and more valuable feedback regarding timely response, effective fire suppression, and efforts to minimize or mitigate smoke and water damage. Asking such questions in a general survey of citizens—​ including customers and noncustomers—​ would yield a high percentage of “no opinion” responses from persons who had not experienced fire services firsthand or would produce a set of naïve responses that would dilute more informed opinions. Similarly, a survey of customers would provide a better basis for judging the effectiveness of recent changes in service delivery than would a survey of all citizens. A general survey of citizens has advantages of breadth and economy. Its results reflect the views of the entire community, including persons who use various facilities and services and those who do not. By combining questions about several local government functions in a single survey, a general citizen survey is much less costly than performing multiple surveys function by function. Furthermore, a survey of citizens can probe for reasons that explain why some respondents do not use various facilities and services—​something a customer survey cannot do.

BOX 28.2  FOCUS GROUPS Conducting a survey is an excellent way to gauge citizen satisfaction generally, but another, somewhat related technique can be even better for securing more detailed insights and advice. Focus groups of 6 to

363

364

o the r analyt ic opt ion s

10 citizens can address service priorities, service proficiency, ideas for improvement, and emerging issues in much greater detail than is possible with a survey of customers or citizens. A focus group session resembles a group interview or structured group discussion in which participants are free to talk with one another. Members of the focus group are asked questions about a service or an issue by a moderator who attempts to probe participants’ attitudes and opinions while encouraging a free flow of ideas through open-​ended discussion. A typical focus group session is 1 to 2 hours in length and participants often are paid a small stipend. A skillful moderator is among the keys to the success of this technique. This person must create an informal environment where participants feel comfortable expressing their opinions. The moderator should be an adept communicator who can direct the conversation without expressing his or her own opinions, rendering judgments on the opinions of others, or ever really becoming a part of the conversation. Furthermore, the moderator must adeptly balance the opportunities for input among the group’s participants. Efforts typically are made to select focus group members who are as representative of the target population as possible given such a small set of people. Success in this regard increases the likelihood that important perspectives will be present in the discussion and enhances the credibility of this qualitative research tool. Almost inevitably, however, even the most carefully selected focus group is unlikely to match the community representativeness of a random sample of citizens carefully drawn for survey purposes. Unlike surveys, which tend to be constrained by a more-​or-​less fixed set of responses for a given service or issue, focus groups allow—​even invite—​ participants to move beyond the boundaries of anticipated responses. Accordingly, the sessions tend to be information-​rich. For example, several local governments have refined their performance measures based on focus group feedback that identified service features citizens value most.

Asking the right questions in the right way Good surveys include good questions that provide useful feedback on important topics. Over time they provide a basis for tracking changes

citiz en s ur v ey s an d c u s t o m e r su rv e y s

in community views and progress in improving government services as viewed by the public. Good questions are unbiased; they do not lead the respondent to a preferred response. Bias can occur with poorly worded questions that fail to reflect neutrality; it can be introduced through the use of ideological labels or terms that might be regarded as pejorative (cops, bureaucrats, and red tape, for example); and it even can occur with patterned response categories that consistently have favored responses at one end of the response range and thereby provide a cue to the respondent. Good questions have response options that are exhaustive (all possibilities are covered) and mutually exclusive (response options are distinct from one another rather than overlapping). Good surveys avoid double-​barreled questions. A single query that actually poses two questions but demands a simple yes or no response is a double-​barreled question. Asking “Do you favor after-​school programs and the criminal prosecution of juveniles to reduce delinquency problems?” with only a single yes or no response option might stump respondents who favor one option but not the other. Pretesting the questionnaire with a small set of persons representing the respondent population allows the survey designer a chance to catch and correct questions with this or other technical problems as well as questions that are simply confusing or otherwise difficult to answer. In addition to questions regarding respondents’ satisfaction with services, their behaviors (“How often have you visited the city’s library in the past 12 months?”), their preferences, and other opinions, questions pertaining to respondents’ attributes often are included in the questionnaire.4 By including a few semipersonal questions in the survey—​ for example, asking respondents to identify their neighborhood, age category, and perhaps their race and broad income category—​analysts later attempting to interpret survey results will be able to detect any important differences in perceptions of service adequacy that may exist from one neighborhood to another or among other relevant characteristics of respondents. But be

4 Because of the reluctance of some respondents to answer questions deemed to be too personal, researchers and analysts designing survey questionnaires are well-​advised to refrain from asking questions regarding personal characteristics or behavior if this information actually is of minimal value to the project.

365

366

other a nalytic options

careful. Questions that are perceived by respondents as too personal may result in refusal to participate and a low response rate for the survey. Skilled survey designers word attribute questions carefully and usually place them at the end of the questionnaire, where the respondents’ investment of time answering earlier questions and desire to protect that investment can work to minimize the likelihood of an adverse reaction.

Accommodating language differences Many communities have populations that have become more diverse in recent years. Some have a substantial number of non-​English-​speaking residents. Achieving survey results in these communities that truly reflect the views of the entire population requires the willingness and ability of survey administrators and interviewers to accommodate principal language differences among respondents. The ability of questionnaire preparers and interviewers to communicate with the major non-​English-​speaking subgroup in the community—​ often Spanish speakers—​ is increasingly important.

Getting expert assistance On the surface, conducting a survey does not seem too complicated. A look beneath the surface, however, reveals a variety of hazards that could damage or even sink the effort. Employing a staff member with survey expertise or taking steps to secure outside assistance can help the local government navigate past these hazards to produce survey results that are valid and representative of the target population. Customer surveys tend to be shorter and simpler than general citizen surveys. Often they are distributed to the entire customer base and require no sampling. If they are conducted regularly, it may be particularly appropriate to consider developing staff expertise for managing and analyzing this form of feedback. Broader surveys tend to introduce greater complexity and more opportunities for error in survey design and administration, leading many cities and counties to seek outside assistance from consultants or nearby universities. When local governments turn to consultants or universities for survey assistance, they usually do so for three reasons: expertise, workload

citiz en s ur v ey s an d c u s t o m e r su rv e y s

management, and objectivity. Government officials recognize that a host of technical issues must be handled properly to avoid jeopardizing the value of the survey results. Securing outside expertise minimizes the risk of error. They also recognize that imposing responsibility on staff members for conducting interviews or training and supervising volunteers can have a major impact on their ability to keep up with current workload. Furthermore, officials recognize that objectivity in survey design and administration is important not only to survey respondents as they answer questions but also to the audience that eventually will receive the survey report. Objectivity is enhanced by enlisting an outside entity to handle at least the most critical or sensitive tasks associated with the survey. Local governments seeking outside expertise for designing and conducting surveys have several options. In addition to various consulting firms that offer survey research services, similar services are sometimes available from nearby universities, either through a survey research center or from individual faculty members. Yet another potential source of expertise is available through a partnership formed by the International City/​County Management Association with the National Research Center.5 This partnership, known as the National Citizen Survey™, was designed to hold down costs and facilitate interjurisdictional comparison of survey results.

Scenario: Back in Idlewild As he explored surveying options, Simon thought about Paula’s suggestion that he get input on programming desires from noncustomers. He also took her advice and visited Andy Jackson, director of the Senior Center. Soon thereafter he discussed his plans with Paula. “I am determined to get big participation numbers in the after-​school program,” he said. “I spoke with Andy Jackson and he’s had good luck using focus groups to identify problems, brainstorm solutions, and generate programming ideas. Before going to the survey option I want to try working with a couple of focus groups—​one made up of kids and the other made up of parents. I plan to have a couple of participants or parents of participants in each focus group, but mostly the groups will consist of nonparticipants in the after-​school program.” 5 Information on the National Citizen Survey is available at www.icma.org/​ncs.

367

368

other a nalytic options

“That sounds like a good plan,” Paula said. “After you’ve made adjustments to the program, I still think we should add a few questions to the city’s general citizen survey that ask about the after-​school program.” “I agree absolutely,” Simon responded. “I just think my current customer surveys and these planned focus groups will give me more detailed programming insights and feedback. Then we can follow up with general citizen survey questions for a broader overall evaluation.”

Suggested for further information Eller, Warren S., Brian J. Gerber, and Scott E. Robinson. Public Administration Research Methods: Tools for Evaluation and Evidence-​Based Practice, 2nd ed. New York: Routledge, 2018. See “Survey Data,” Chapter 11, 221–​241. Fink, Arlene. The Survey Handbook, 2nd ed. Thousand Oaks, CA: Sage Publications, 2003. Hatry, Harry P., Donald M. Fisk, John R. Hall Jr., Philip S. Schaenman, and Louise Snyder. How Effective Are Your Community Services? Procedures for Performance Measurement. 3rd ed. Washington, DC: International City/​ County Management Association and The Urban Institute, 2006. Hatry, Harry P., John E. Marcotte, Therese van Houten, and Carol H. Weiss. Customer Surveys for Agency Managers: What Managers Need to Know. Washington, DC: Urban Institute Press, 1998. Krueger, Richard A., and Mary Anne Casey. “Focus Group Interviewing.” In Handbook of Practical Program Evaluation, 4th ed., edited by Kathryn E. Newcomer, Harry P. Hatry, and Joseph S. Wholey, 506–​534. Hoboken, NJ: Jossey-​Bass, 2015. Miller, Thomas I., and Michelle Miller Kobayashi. Citizen Surveys: How to Do Them, How to Use Them, What They Mean, 2nd ed. Washington, DC: ICMA, 2000. Newcomer, Kathryn E., and Timothy Triplett. “Using Surveys.” In Handbook of Practical Program Evaluation, 4th ed., edited by Kathryn E. Newcomer, Harry P. Hatry, and Joseph S. Wholey, 344–​382. Hoboken, NJ: Jossey-​ Bass, 2015.

29 ANALYZING SURVEY DATA REVEALING GRAPHICS AND SIMPLE STATISTICS

After the data from a carefully designed and administered survey are collected, the focus moves to the analysis of findings and the reporting of results. What analysis and reporting options does the manager or analyst have?

Scenario: Tillery, Colorado The budget staff in Tillery, Colorado, meets each week, mostly to discuss budget procedures, problems, and forecasts. Occasionally the conversation shifts to budgetary maneuvering by various departments or to predicted council stances on tax increases in the upcoming year. Today’s discussion departed from the usual set of topics. “We’ve been conducting citizen surveys in Tillery for a long, long time, and I am proud of that,” said Eve Bauer, the city’s budget director. “Because we’ve been doing them so long, the city of Tillery has gained a reputation as a leader in citizen surveying. That’s flattering, but, to be honest, I think

370

other a nalytic options

we could be criticized for how little we use the information. We don’t go as far as we should in the analysis of our survey data.” Nick Wilson and Maria Pickwick, the two budget analysts who had worked most closely with the survey data in the past, responded quickly and a bit defensively. “We report the results for each item,” Wilson said, “so we know whether most folks are satisfied with a given service or not.” “And we always include tables that show whether the percentages are more or less favorable than in previous years,” added Pickwick. “Departments know whether public regard for their programs is moving in the right direction.” “That’s absolutely correct,” replied Bauer. “I’m not saying we are doing a bad job of conducting the survey or a bad job of reporting the most fundamental results. I am simply saying that we could step up to the next level of analysis without a great deal of effort. We might learn something more from the data if we do so.” Wilson and Pickwick agreed to explore options for enhancing the analysis of the citizen survey. Bauer encouraged the analysts to investigate what other local governments were doing in the analysis of survey data and urged them to focus primarily on the use of more revealing graphics and simple statistics that could be explained easily to department heads, council members, and other readers of the survey report. “I am looking for tools that will help us understand and explain to others what the citizens are telling us. The simpler the better, as long as it does the job well.”

More revealing graphics Summary graphs and statistics can mask a lot of important details. Knowing, for example, that 60 percent of all Tillerians feel “safe” or “very safe” walking alone in their neighborhoods at night makes Tillery seem like a pretty secure place. But what if that 60 percent figure is based on an 85 percent affirmative response from males and only 35 percent from females? If barely more than one-​third of Tillery’s women feel safe walking alone in their neighborhoods at night, would this discovery prompt new programs to reduce their vulnerability or their level of anxiety? What if the overall 60 percent security level hides remarkable variation among neighborhoods, some reaching as high as 90 percent and others as low as 30 percent?

a n aly z i n g s u rv e y data

Would this discovery lead to more focused police efforts and higher visibility in targeted areas? Often, survey results such as the ones noted in the preceding paragraph are depicted in tables called “cross-​tabulations” or simply “cross-​tabs.” One set of variables is displayed in the rows and another set in the columns. For example, a two-​by-​two cross-​tab for the case presented above would contain four cells, with separate cells showing the number of males feeling safe, males feeling unsafe, females feeling safe, and females feeling unsafe walking alone in their neighborhoods at night.1 By producing tables and graphs that depict key distinctions in the responses of different groups of citizens, an analyst might spur important new strategies in a community at a time when changes are needed. In contrast, by producing only a summary graph that camouflages these differences an analyst could inadvertently and inappropriately rationalize continuation of the status quo. The city of Portland, Oregon, regularly surveys a large sample of residents to gauge their views on municipal services and their satisfaction with Portland as a place to live, raise children, and work or go to school. In a recent survey 50 percent of all respondents said that they had participated in a local parks and recreation program during the previous year. If city officials considered only this overall percentage, some important differences in program participation rates that were revealed only when analysts disaggregated responses by ZIP codes would have been lost (Figure 29.1). Portland’s use of a map to depict differences across neighborhoods shows that displays other than tables can be very effective ways to draw attention to an important point. Tables, however, are more common. For example, the city of Takoma Park, Maryland, used a table with rows, columns, and shading to show important differences in the use of community amenities by ethnicity and race (Table 29.1). When the responses of different groups of respondents almost mirror one another and, therefore, differ little from the overall average, a single summary graph or table will suffice. In fact, a single graph in such cases, with a note stating that no differences across subgroups were detected, is preferable to multiple graphs that shed no new light on the topic. When

1 For an example of a cross-​tabulation, see Table 29.4.

371

newgenrtpdf

372 other a naly t ic options

Figure 29.1  Rate of Participation in Portland (Oregon) Parks and Recreation Programs Varies by Residents’ ZIP Codes Source:City of Portland (Oregon), 2019 Portland Insights Survey:Final Report (Portland, OR: City of Portland Budget Office and HR&A, August 2019/​Updated October 2019), 57. Image courtesy of HR&A Advisors, Inc.

a n aly z i n g s u rv e y data

Table 29.1  Differences by Ethnicity and Race in the Use of Community Amenities in Takoma Park, Maryland: Survey Results In the last 12 months, about how many times have you or other household members done the following things: Percentage doing at least once in last 12 months

Hispanic

Non-​ Hispanic

White

Black

Other race or 2+ races

Overall

Used a public computer at the Takoma Park, Maryland Library or at the adjacent Computer Center Used Wi-​Fi connections in a Takoma Park City government facility Visited a park or trail within the City Taken your dog to the dog park Received financial assistance for a housing or other emergency Used Recreation Department services Received a scholarship from the Recreation Department Rode a dockless bike Used public transportation Received a “municipality letter” for a building permit

21%

28%

28%

34%

17%

27%

16%

27%

35%

14%

15%

26%

80% 29% 3%

87% 19% 5%

95% 24% 3%

67% 12% 11%

93% 23% 2%

86% 20% 5%

33% 0%

47% 3%

54% 0%

33% 6%

39% 6%

45% 2%

9% 83% 17%

16% 88% 9%

22% 93% 12%

5% 78% 4%

13% 86% 15%

15% 88% 10%

Source: City of Takoma Park, Maryland, The 2018 City of Takoma Park Resident Survey: Report of Results (Boulder, CO: National Research Center, 2019), 213. Reprinted by permission.

responses vary sharply from one group of respondents to another, however, the absence of tabular or graphic displays depicting these differences would be a serious breach of thoroughness in staff work. An analyst performing thorough staff work will alert the audience to such significant differences.

Pertinent statistics The most common statistics reported with local government surveys are descriptive statistics, such as proportions and measures of central tendency (especially the mean or median—​see Chapter 2). These statistics are simple to calculate and easy to explain to the audience of the survey report. Unfortunately, they do not always reveal as much as the analyst needs to show.

373

374

other a naly t ic options

Although descriptive statistics present clear evidence about the characteristics or views of a sample of respondents, a perceptive audience may wonder how well they represent the views of the general population. When the sample is large and the results are lopsided, most people are comfortable making an intuitive inference. For instance, if 90 percent of the 500 male respondents to a survey and only 20 percent of the 500 female respondents said they would support the establishment of a rugby league in the city’s recreation program, the analyst would probably be confident in reporting a clear gender split on this topic in the general population. Intuitively, the analyst knows that a split so stark in a sample this large is likely to reflect a similar division in the population as a whole. But what if the split were less pronounced, with perhaps a support level of 60 percent among men and 50 percent among women? Although it would be accurate to declare that men in this sample were more supportive than women in the sample, would the analyst be foolish to declare that men in the community are more supportive than women of a new rugby league, when the division of responses is so narrow? This is a common occurrence in the analysis of survey data and a problem that another simple statistic can help to solve. Chi-​square is a statistic used in the analysis of categorical or nominal data (for example, data that may be categorized as men or women, recipients of a particular service or nonrecipients, city residents or visitors, satisfied with the service or dissatisfied, and so on). The chi-​square statistic helps the analyst learn whether respondents in one category (for example, men) are significantly different in a statistical sense from those in another category (for example, women) with regard to another set of nominal data (for example, satisfied or dissatisfied with a given service). The chi-​square (depicted symbolically as X2) is calculated using the following formula: X = 2

∑

( fo − fe )2 fe

where fo and fe refer respectively to the observed and expected frequencies for each cell in a table presenting one set of nominal data as rows and another set as columns. Suppose 50 men and 50 women were interviewed, with 60 respondents favoring a new rugby league and 40 opposing it. Because half of the respondents are men, it might be expected that half of the proponents (that is,

a n aly z i n g s u rv e y data

Table 29.2  Support for Proposed Rugby League among Sample of Citizens Gender of respondent Female Male Total

Favor

60

Oppose

Total

40

50 50 100

Table 29.3  Calculating the Expected Frequency of Cells from Knowledge of the Marginals Gender of respondent

Favor

Oppose

Total

Female Male Total

fe = 30 fe = 30 60

fe = 20 fe = 20 40

50 50 100

Note: fe is the expected frequency for a given cell based on the proportions found in the marginal. Calculation for a given cell is: fe = (row marginal/​N) × column marginal. For example, fe for upper left cell = (50/​100) × 60 = 30.

30) and half of the opponents (that is, 20) would be men. These expected frequencies can be calculated by constructing a table for the variables (see Table 29.2), focusing only on the totals for the rows and columns (known as “marginals”), and applying the proportions for rows to the totals for columns to calculate an expected frequency for each cell.2 If the proportion of women is 50 out of 100 respondents (50/​100) and that proportion is applied to the “favor” column, the result is an expected frequency of 30 in the female-​favor cell (50/​100 x 60 = 30) and 30 in the male-​favor cell (see Table 29.3). Now, suppose that the observed frequencies (the actual responses, as shown in Table 29.4) differ from the expected frequencies. The chi-​square statistic can be used to determine if the findings are statistically significant—​ that is, if they are sufficiently different from expected frequencies to conclude that differences in one variable in the population as a whole (for example, support for a rugby league) are likely to be related to differences in the other (for instance, gender). In the case of the rugby survey, the calculations of the various elements in the formula yield a chi-​square of 2 Alternatively, the proportions for columns could be applied to the totals for rows to arrive at the same results.

375

376

other a naly t ic options

Table 29.4  Actual Support (“Observed Frequencies”) for Proposed Rugby League among a Sample of Citizens Gender of respondent

Favor

Oppose

Total

Female Male Total

23 37 60

27 13 40

50 50 100

Table 29.5  Calculations for Chi-​Square Cell

fo

fe

fo − fe

(fo –​fe)2

(fo –​fe)2 /​fe

a b c d Total

23 27 37 13 100

30 20 30 20 100

−7 7 7 −7

49 49 49 49

1.633 2.450 1.633 2.450 8.166

8.166 (see Table 29.5). The value of chi-​square is said to be “statistically significant” only if it exceeds the relevant value shown in a chi-​square table (see Appendix A). Calculating the chi-​square yields one of the three things needed in order to use the chi-​square table. The second thing needed is to decide how rigorous the test should be—​in other words, how small the probability (p) of a mistaken verdict should be (for example, a p of 0.01 imposes a more stringent test than a p of 0.05). A p of 0.01 will be used here. Finally, the degrees of freedom in the table of survey results must be determined. Degrees of freedom (df) can be calculated using the following formula: df = ( r − 1) ( c − 1) where r is the number of rows in the table and c is the number of columns. In the example, the table has two rows (female and male) and two columns (favor and oppose), which yields only 1 degree of freedom.3 3 If the marginals (row and column totals) and enough of the cell entries are known in a given table, the values for all the remaining cells in that table can be filled in. Degrees of freedom indicate how many cells have to be known in a given table, along with the marginals, in order to be able to fill in the rest of that table. In the case of a table with two rows and two columns, knowing just one cell value (df = 1) along with the row and column totals is all the information needed to complete the table.

a n aly zi ng s u rv e y data

Armed with these three pieces of information (X2 = 8.166, p = 0.01, and df = 1), the chi-​square table can be used to determine that the results are statistically significant at the .01 level only if the chi-​square is greater than or equal to 6.635. Because the value in the example (X2 = 8.166) exceeds that level, it can confidently be asserted that males in this community are more favorably inclined toward a community rugby league than are females. The odds are great that these two sets of variables (that is, gender and support for a rugby league) are related to each other and that the survey results did not simply occur by chance. The odds that the data distribution in this table could have occurred simply by chance, rather than because of a relationship between variables, are less than 1 percent (p < .01).

FROM THE ELECTRONIC TOOLKIT BOX 29.1  TESTING FOR STATISTICAL SIGNIFICANCE The Excel spreadsheet will test for statistical significance and report probabilities based on chi-​square tests of data distributions. Unfortunately, however, Excel does not report the chi-​square value itself. To use this significance-​testing function, enter the sets of data for the actual frequencies and the expected frequencies for a cross-​ tabulation into the spreadsheet. (For directions on opening Excel and entering data refer to Box 1.1.) An analyst examining the rugby example in this chapter would enter the expected frequency of men and women who were opposed to or in favor of a rugby league as well as the actual support for the league from the sample observations. After the data are entered, type CHITEST and enter the cell names of the actual range and the expected range of frequencies in parentheses after the command. For example, if the actual data were entered in chart form in cells B2 through C3 and the expected data were entered in cells B6 through C7, then the intern would enter “=CHITEST(B2:C3,B6:C7)” in any cell. For Excel 2007 users, an alternative to typing this instruction would be to choose “Formulas” from the menu options at the top of the screen, and select the “CHITEST” function from the statistical function option under “More Functions.” This method includes a prompt for the actual and expected ranges needed in the formula. For older versions

377

378

other a naly t ic options

of Excel, use the Function choice from the Insert menu. The symbol for this choice (fx) also appears on the toolbar. The output indicates the probability—​in this case, .004267—​that the results occurred by chance. It does not show the actual chi-​square statistic that would be found if using a statistics package rather than a spreadsheet or if doing the calculations by hand. Excel does, however, report the probability that the user would otherwise have to search for on a chart. Because the probability reported in this example (.004267) is very small and, more important, is less than the specified .02 probability mark, the result is said to be statistically significant. There is little likelihood that the relationship depicted in this cross-​tabulation (see Table 29.4) could have happened by chance.

Chi-​square can be a valuable analytic tool for exploring relationships between variables that are nominal or categorical in nature as long as the sample categories are large enough for this statistic to be applicable (that is, the table should have an expected frequency in each cell of 6 or greater). Other easy-​to-​apply statistics that are often useful in the analysis of citizen surveys are rank-​order correlations, which are applicable when respondents are asked to rank preferences, and t-​tests, which allow the analyst to assess whether differences in the average response of two groups are statistically significant. Details on rank-​order correlations will not be provided here but may be found in most statistics textbooks. T-​tests are addressed in Chapter 9 “Detecting meaningful differences in average performance.”

FROM THE ELECTRONIC TOOLKIT BOX 29.2  ON-​LINE CHI-​SQUARE CALCULATORS Chi-​square calculators are posted on the web from time to time and may be located by use of any search engine. The link for a recently posted and useful calculator is: www.physics.csbsju.edu/​stats/​contingency_​NROW_​NCOLUMN_​ form.html

an aly z i n g su rv e y data

Back in Tillery Analysts Nick Wilson and Maria Pickwick explored how other local governments were analyzing survey data and using graphs to depict important findings. They also brushed up on chi-​square and a few other basic statistics and began to look at their own survey reports in a new light. “I used to think our analysis was more thorough and our reports more impressive and useful than I do now,” Wilson confessed. “In fact, I thought Eve was off base when she asked us to look into this. But not anymore.” “I know what you mean,” replied Pickwick. “My feeling now is that we can do a lot more with the great data we get from the survey. I think our next report can be much better than the last one, even by incorporating some of the simple techniques we’ve picked up in the last few days.”

AVOID THIS COMMON ERROR BOX 29.3  “PERCENTAGE” OR “PERCENTAGE POINTS”? All too often, analysts misrepresent their findings by saying or writing “percentage” when they actually mean “percentage points.” Percentage (sometimes the word “percent” is used instead) and percentage points are two very different things. For example, an increase from 2 percent to 3 percent is not a 1 percent increase. This change should be reported either as a one-​percentage-​point increase or as a 50 percent increase. An increase from 30 percent to 50 percent is a sizable jump that correctly may be depicted either as an increase of 67 percent or 20 percentage points. It is misrepresented if the analyst reports the growth to be only 20 percent. Consider a pair of surveys conducted in consecutive years, each having 100 respondents. In the first survey, 8 percent of the respondents (8 persons out of 100) reported having visited the brand-​new “tot lot” at the city’s Central Park; but a year later, 15 percent of the respondents to the second survey (15 persons) said they had been there. If the analyst simply subtracted the first result from the second and declared this to be growth of 7 percent in the use of the new facility (8 percent to 15 percent), the analyst would be seriously understating the actual rate of growth. Increasing from 8 respondents to 15 respondents visiting

379

380

othe r analy tic op tion s

the tot lot—​almost a doubling of the number from the respective pools of 100 respondents—​is an increase of 87.5 percent! The analyst would have been correct to report either an 87.5 percent increase or a seven-​ percentage-​point increase—​but not an increase of 7 percent.

Before meeting again with Budget Director Bauer to share the results of their investigation, the analysts decided to develop a prototype for next year’s survey report by using their new tools on a small portion of last year’s survey results. In most cases, they pursued a four-​part strategy: 1. Examine results overall and also by relevant categories of respondents. 2. Develop cross-​tabulations that compare results by relevant categories, and check to see whether the differences are statistically significant. 3. Do not commit information overload by including all of these cross-​ tabulations in the report. If none of the cross-​tabulations produces statistically significant results, then report only the overall results for each major topic or issue. 4. If significant relationships are detected, report them in a manner that commands the attention they deserve. The prototype report they prepared included a dozen graphs and tables, including those shown here as Figures 29.2 and 29.3. The budget director

Figure 29.2  Perceived Housing Availability for Various Household Income Ranges in Tillery (in percentages)

a n aly zi ng s u rv e y data

Figure 29.3  Perceived Magnitude of Speeding Problem, by Neighborhood in Tillery (in percentages)

was pleased. “This is just the sort of analysis I had in mind,” Bauer said. “Not only does it give us a more informative report, but I predict that some of these breakdowns by neighborhood will actually influence some operating strategies. Then we can see whether these strategies begin to improve the effectiveness and equity of our services. In my book, we are finally beginning to see the real value of citizen surveys.”

AVOID THIS COMMON ERROR BOX 29.4  CAVALIERLY COMPARING SCORES ACROSS LOCAL GOVERNMENT FUNCTIONS In reporting survey results, local government officials often compare the levels of citizen satisfaction expressed for various municipal or county functions. This ranking of satisfaction scores may be taken to imply that managers and employees in favored departments are doing better than those in departments earning lower satisfaction scores. Perhaps they are; but not necessarily. It is important to recognize that some functions—​e.g., the fire department and library—​almost invariably receive high satisfaction ratings in community after community, while other functions—​e.g., street maintenance—​seem doomed to lower ratings. It is as if each function operates in its own satisfaction range, with “excellence” for some functions earning a score approaching 100 and “excellence” for others somewhat lower.

381

382

other a nalytic options

This is why some survey experts recommend comparing the scores for each function with its counterparts performing the same function in other communities rather than with those performing different functions in the same community.1 1 Thomas I. Miller and Michelle A. Miller, “Standards of Excellence: U.S. Residents’ Evaluations of Local Government Services,” Public Administration Review 51, No. 6 (November/​December 1991): 503–​514.

BOX 29.5  OTHER APPLICATIONS OF THE CHI-​SQUARE STATISTIC In addition to its usefulness in detecting and reporting statistically significant survey results, the chi-​square statistic can perform the same service for other types of nominal data collected by means other than surveys. Consider, for instance, the debate in a given community over the qualification requirements for new police officers or the effectiveness of a tuition reimbursement program as a retention device for employees in general. Is the evidence that rookie officers with college degrees are more likely to complete probation and less likely to be the target of citizen complaints statistically significant? Are employees who participate in the tuition reimbursement program more likely than nonparticipants to remain with the local government at least five years? Because these questions involve nominal (categorical) variables—​that is, college degreed versus nondegreed candidates, completion versus noncompletion of probation, recipient versus nonrecipient of citizen complaints, tuition program participant versus nonparticipant, employment of five years or more versus employment of a shorter duration—​each can be analyzed with cross-​tabulations of data that are probably on hand and answered with the chi-​square statistic. The same applies to questions involving nominal data and other local government issues.

References City of Portland (Oregon). 2019 Portland Insights Survey: Final Report. Portland, OR: City of Portland Budget Office and HR&A Advisors, Inc., 2019. City of Takoma Park (Maryland). The 2018 City of Takoma Park Resident Survey: Report of Results. Boulder, CO: National Research Center, Inc., 2019.

a n aly z i n g s u rv e y data

Suggested for further information Eller, Warren S., Brian J. Gerber, and Scott E. Robinson. Public Administration Research Methods: Tools for Evaluation and Evidence-​Based Practice, 2nd ed. New York: Routledge, 2018. See “Hypothesis Testing,” Chapter 15, 311–​316. Folz, David H. Survey Research for Public Administration. Thousand Oaks, CA: Sage Publications, 1996. Gibilisco, Stan. Statistics Demystified, 2nd ed. New York: McGraw-​Hill, 2011. Hatry, Harry P., John E. Marcotte, Therese van Houten, and Carol H. Weiss. Customer Surveys for Agency Managers: What Managers Need to Know. Washington, DC: Urban Institute Press, 1998. Miller, Thomas I., and Michelle Miller Kobayashi. Citizen Surveys: How to Do Them, How to Use Them, What They Mean, 2nd ed. Washington, DC: ICMA, 2000. Newcomer, Kathryn E., and Timothy Triplett. “Using Surveys.” In Handbook of Practical Program Evaluation, 4th ed., edited by Kathryn E. Newcomer, Harry P. Hatry, and Joseph S. Wholey, 344–​382. Hoboken, NJ: Jossey-​ Bass, 2015. Sapsford, Roger, and Victor Judd. Data Collection and Analysis. 2nd ed. Thousand Oaks, CA: Sage Publications, 2006.

Web resources College of Saint Benedict & Saint John’s University, “Tools for Science” www. physics.csbsju.edu/​stats/​contingency_​NROW_​NCOLUMN_​form.html Mathcracker, “Chi-​Square Test of Independence” https://​mathcracker.com/​ chi-​square-​test-​of-​independence Preacher, Kristopher J. “Calculation for the Chi-​Square Test” http://​quantpsy. org/​chisq/​chisq.htm Stangroom, Jeremy. “Chi-​Square Test Calculator” www.socscistatistics.com/​ tests/​chisquare2/​default2.aspx For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

383

30 FINANCIAL CONDITION ANALYSIS

For a local government to continue delivering services, it must have resources sufficient to pay for the salaries, supplies, equipment, buildings, and other necessary expenses associated with operating its departments and programs. Financial health is essential for the viability of these operations over the long term. An organization’s ability to make appropriate fiscal decisions and necessary adjustments to sustain its operations is dependent not only on the financial knowledge and skills of its managers but also on their conscientiousness in monitoring and responding to changes in financial condition.

Scenario: Hutchison, Iowa City manager Lily Mellis and finance director Dan Schaffner were sitting in Lily’s office early one Tuesday morning, discussing Monday night’s city council meeting. The city staff had provided each council member a copy of the city’s Annual Financial Report prior to a presentation by the city’s

f in a n c i al c o n di t i o n a n aly s i s

auditor at the meeting. The auditor made several preliminary remarks about the mechanics of conducting a financial audit and then declared this to be “a clean audit,” with no substantial findings to report. One of the council members took the clean audit declaration to mean that the city was in good financial condition and expressed his pleasure over that finding. The auditor quickly corrected him, saying that the audit’s purpose was only to verify that the financial report could be relied upon as an accurate statement. He and his colleagues at the audit firm, he said, were not making a claim about the city’s financial health. The discussion did not go much further, but the comment worried Lily. “I’m concerned, Dan, that some council members—​perhaps even most of them—​may not have a good grasp of the information in our financial report and understand what the numbers are telling them. Your department did a really good job pulling the report together. But the report is long and most of our council has had little experience reading financial statements.” “But I think we put together a really good annual financial statement” replied Dan. “We’ve won an award two years running now from the Government Finance Officers Association (GFOA) for our financial reports. We’re following legal requirements and best practices.” “Dan, I’m not criticizing what you’ve done,” assured Lily. “My question is whether we may need to do some additional work to better communicate with the council and citizens. I’m thinking of something that might be less overwhelming than a full financial statement—​something that focuses more concisely on the big picture and several key financial components. It needs to address the city’s financial health in a way that will provide important background for any important fiscal choices that lie ahead.” She explained the need for a more concise set of facts and figures by saying, “I think it’s a little like when I go to my doctor for my annual physical and get several pages of test results back. The report may be accurate, but I’m not always sure what it all means. I fear that we are doing something like that when we give city council members our annual financial statement and expect them to figure out our financial condition from that.” Dan nodded in agreement with her doctor’s visit analogy. “Let me work with my team and see if we can find a good way to get at your concern. Maybe we can create a better patient’s report on Hutchison’s financial health.”

385

386

other a nalytic options

Options for financial condition analysis Over time several models have been put forth designed to help individual local governments conduct their own financial health assessment. Among them are the following: • • • • • •

the International City/​County Management Association (ICMA) model the Ives and Schanzenbach assessment tool the Governmental Accounting Standards Board (GASB) assessment tool the Brown 10-​point test Mead’s 10-​point test the North Carolina model

Perhaps the best known framework for analyzing the financial condition of a local government is one developed for ICMA.1 This longstanding model was established on the premise that financial condition is tied to cash solvency, budgetary solvency, long-​run solvency, and service-​level solvency.2 Relying on indicators addressing 42 separate aspects of financial condition, the ICMA model has the benefit of comprehensiveness, but its use involves a data-​collection task and level of complexity greater than some local governments have been willing to tackle. Similar complexity is found in another pair of assessment tools. One developed by Martin Ives and Kirk Schanzenbach addresses cash solvency, structural budgetary solvency, long-​term solvency, economics and demographics, and other factors through a set of 19 indicators.3 Another, offered by the Governmental Accounting Standards Board (GASB), directs analysts to 29 ratios when user guides and follow-​up articles are considered altogether.4

1 Karl Nollenberger, Sanford M. Groves, and Maureen Godsey Valente, Evaluating Financial Condition: A Handbook for Local Government, 4th ed. (Washington, DC: International City/​ County Management Association, 2003). 2 Sanford M. Groves, W.M. Godsey, and M.A. Shulman, “Financial Indicators for Local Government,” Public Budgeting & Finance 1, no. 2 (Summer 1981): 5–​19. 3 Martin Ives and Kirk Schanzenbach. Financial Condition Analysis and Management. Fort Worth, TX: Sheshunoff Information Services, 2001. 4 D.M. Mead, An Analyst’s Guide to Government Financial Statements (Norwalk, CT: Governmental Accounting Standards Board, 2001). Also see D.M. Mead, “Assessing the Financial

f in a n c i al c o n di t i o n a n aly s i s

To counter the complexity of these other financial condition assessment tools, Ken W. Brown in the mid-​1990s proposed relying on a smaller set of 10 ratios of financial condition.5 His 10-​point test gained considerable popularity for its simplicity and usefulness. Dean Michael Mead, a contributor to GASB’s more elaborate analytic model described above, has also offered a simpler assessment tool.6 Mead’s tool builds upon Brown’s 10-​point test, retaining the original test’s simplicity but incorporating other information to address long-​term aspects of financial condition as well as the short-​term aspects emphasized in Brown’s original. Mead’s model relies in part on information now being reported by local governments complying with GASB’s reporting requirement known as “Statement 34.”7 Statement 34 led to changes in local government reporting that enabled governmental activities to be expressed in terms of a full accrual model, giving a view of the full economic resources of governmental funds. Like Brown’s original, Mead’s 10-​point test relies on ratios calculated from information commonly found in standard financial reports. These ratios assess short-​run financial position, liquidity, financial performance, solvency, revenues, debt burden coverage, and capital assets. Both Brown and Mead encouraged local government officials to use their assessment tools to compile key financial ratios not only for their own government but also for a set of comparable governments. Governments following this prescription could assess their financial standing within the group. Unfortunately, the collection of this much data about their own government and others has been perceived to be a burden greater than many governments have been willing to tackle.

Condition of Public School Districts,” in Selected Papers in School Finance, 2000–​01, ed. W.J. Fowler (Washington, DC: US Department of Education, National Center for Education Statistics, 2001), 55–​76; and B.A. Chaney, D.M. Mead, and K.R. Schermann, “The New Governmental Financial Reporting Model,” Journal of Government Financial Management 51, no. 1 (Spring 2002): 26–​31. 5 Ken W. Brown, “The 10-​ Point Test of Financial Condition: Toward an Easy-​ To-​ Use Assessment Tool for Smaller Cities,” Government Finance Review (December 1993): 21–​26. 6 Dean Michael Mead, “A Manageable System of Economic Condition Analysis for Governments,” in Public Financial Management, ed. Howard A. Frank (New York: Taylor & Francis, 2006), 383–​419. 7 ”Statement No. 34,” in Basic Financial Statements—​and Management’s Discussion and Analysis—​for State and Local Governments (Norwalk, CT: Governmental Accounting Standards Board, 1999).

387

388

other a nalytic options

Another financial condition model—​this one developed by researchers at the University of North Carolina’s School of Government—​emphasizes data accessibility and ease of use.8 This model relies on a small set of key ratios, using data typically found in the financial statements of local governments. Like Brown and Mead’s tests, the model encourages a government to compile comparison information for at least three peer governments.

Hutchison’s search for an assessment tool Finance director Dan Schaffner asked his top analyst, Chaz George, to look for some practical techniques for measuring a local government’s financial condition. In his search, Chaz reviewed several models, including the six highlighted above. When he came across the North Carolina model, he liked what he saw. Although it might not be quite as comprehensive as some of the others, it touched all the most important bases, in his opinion, and the required data would be easy to collect—​not only for Hutchison but for the comparison governments as well. When he met with Dan, Chaz briefed his boss on all the models and carefully described the pros and cons of each. He mentioned the data collection burden repeatedly. “So, what do you recommend?” Dan asked. “The North Carolina model offers a nice balance,” Chaz replied. “It would give us an assessment almost as comprehensive as the others, and the time and effort required to do the analysis will probably be less than for most—​and perhaps all—​of the other tools. The data we’ll need is right in the financial statements. I think we ought to try that one.” Dan agreed, so they decided to try out the North Carolina model, initially just pulling the data needed in order to do the governmental activities financial condition assessment for Hutchison and for three other cities. A few days later, Chaz was ready to report on his trial run.

8 William C. Rivenbark, Gregory S. Allison, and Dale J. Roenigk, “Conceptualizing Financial Condition in Local Government,” Journal of Public Budgeting, Accounting, and Financial Management 22, no. 2 (Summer 2011): 241–​267.

f in a n c i al c o n di t i o n a n aly s i s

The North Carolina model The North Carolina model accommodates both modified accrual accounting (typically used for a local government’s general fund) and full accrual accounting for governmental activities (all governmental funds combined) and business activities (enterprise funds such as water and sewer). The model presents financial measures broken into two categories: flow and stock. Flow covers data from the operations statements and shows flows of revenues and expenses over the course of the fiscal year. Stock focuses on assets and liabilities at a point in time, traditionally the last day of the fiscal year. The six measures in the model prescribed for the general fund are shown in Tables 30.1 and 30.2 with explanations and calculation notes. Additionally, a set of eight measures is prescribed for governmental and business activities beyond the general fund that are accounted for on a full accrual basis. These measures are shown in Tables 30.3 and 30.4. Although there is overlap in the dimensions measured for general fund and governmental activities, the financial condition ratios can differ as the first relies on modified accrual and the latter uses full accrual accounting. The model calls for five years of data not only for one’s own government but also for at least three peer governments. Because the needed data can be found in annual financial statements and these are commonly posted online, the projected time requirement for gathering data is approximately 1 hour per jurisdiction per year of data. This means that the first year the model is used, data collection for the full set of governments might take 20 to 30 hours. Updating the data in subsequent years should take only 4 to 6 hours of time.9

Back in Hutchison “Here’s our trial run,” Chaz explained as he spread a few sheets of paper in front of Dan. “Remember, this is not the whole model. I just pulled enough data to do the governmental activities financial condition 9 After testing and demonstration, the North Carolina Treasurer’s Office put the model in place online as part of its oversight of local government finances. All 100 North Carolina counties and more than 540 municipalities in the state have access to these dashboards with a few clicks on the website.

389

newgenrtpdf

390

Table 30.1  Financial Flow Indicators for General Fund Description

Financial indicator

Calculation

Data source

Interpretation

Service obligation

Addresses whether or not a government’s annual revenues were sufficient to pay for annual operations

Operations ratio

Total revenues divided by total expenditures (plus transfers to debt service fund and less proceeds from capital leases and installment purchases)

Statement of revenues, expenditures, and changes in fund balances

Dependency

Addresses the extent to which a government is reliant on other governments for resources

Intergovernmental ratio

Total intergovernmental revenue divided by total revenue (This includes restricted and unrestricted revenues.)

Statement of revenues, expenditures, and changes in fund balances

Financing obligation

Provides feedback on service flexibility, as affected by the amount of expenditures committed to annual debt service

Debt service ratio

Debt service (principal and interest payments on long-​term debt, including transfers to the debt service fund) divided by total expenditures plus transfers to debt service fund (minus proceeds from capital leases and installment purchases)

Statement of revenues, expenditures, and changes in fund balances

Ratio at or above 1.0 means the government lived within its financial means. A ratio a little above 1.0 allows for capital replacement. Ratios below 1.0 indicate a deficit and cannot be sustained indefinitely. A high ratio indicates heavy reliance on other governments and possible vulnerability to an unreliable revenue stream. A high ratio threatens the availability of resources for service delivery. Ratios above 20 percent may be particularly risky.

Source: Adapted from William C. Rivenbark, Dale J. Roenigk, and Gregory S. Allison, “Communicating Financial Condition to Elected Officials in Local Government,” Popular Government 75, no. 1 (2009): 10. Used by permission.

other a naly t ic options

Financial dimension

newgenrtpdf

Table 30.2  Financial Stock Indicators for General Fund Description

Financial indicator

Calculation

Data source

Interpretation

Liquidity

Government’s ability to meet short-​term obligations

Quick ratio

Balance sheet

A low ratio may warn of difficulty in meeting the government’s short-​term obligations.

Solvency

Government’s ability to address long-​term obligations

Fund balance as a percentage of expenditures

Cash and investments divided by current liabilities (not including deferred revenue) Available fund balance divided by total expenditures (less proceeds from capital leases) plus transfers out

Balance sheet

Leverage

Extent to which a Debt as a government relies on percentage of tax-​supported debt assessed value

Tax-​supported, long-​term debt divided by assessed value

Debt management reports and statistical notes

An especially low ratio may warn of difficulty in addressing revenue fluctuations and the government’s longer-​term obligations. A minimum of 8 percent (roughly one month’s expenditures) is typically recommended. A high ratio suggests excessive debt. Some states set statutory upper limits.

Source: Adapted from William C. Rivenbark, Dale J. Roenigk, and Gregory S. Allison, “Communicating Financial Condition to Elected Officials in Local Government,” Popular Government 75, no. 1 (2009): 11. Used by permission.

f in a n c i al c o n di t i o n a n aly s i s

Financial dimension

391

392

Financial dimension

Description

Interperiod equity

Addresses Total whether a margin government ratio lived within its financial means during the year

Financial Measures performance change in government’s financial position as a result of resource flow

Financial indicator

Percentage change in net assets

Governmental activities

Business activities (enterprise funds)

Interpretation

Calculation

Data source

Calculation

Data source

Total resource inflow (program revenues plus total general revenues and net transfers) divided by total resource outflow (total expenses)

Statement of activities

Statement of revenues, expenses, and changes in fund net assets

A ratio at or above 1.0 means the government lived within its financial means. A ratio a little above 1.0 allows for capital replacement. Ratios below 1.0 indicate a deficit and cannot be sustained indefinitely.

Change in net assets divided by beginning net assets

Statement of activities

Total resource inflow (operating and nonoperating revenues plus transfers in) divided by total resource outflow (operating and nonoperating expenses plus transfers out) Change in net assets divided by beginning net assets

Statement of revenues, expenses, and changes in fund net assets

A positive percentage indicates an improvement and a net surplus of revenues over expenses this year.

other a naly t ic options

Table 30.3  Financial Flow Indicators for Governmental Activities and Business Activities

newgenrtpdf

Addresses the Charge-​to-​ extent to expense which service ratio charge revenues covered total expenses

Charges for services (fees, fines, charges for service) divided by total expenses

Statement of activities

Charges for services divided by operating and nonoperating expenses

Statement of revenues, expenses, and changes in fund net assets

Financing obligation

Addresses service flexibility, as affected by the amount of expenses committed to debt service

Debt service (principal and interest payments on long-​term debt) divided by total expenses plus principal

Statement of activities (Principal is found in the notes.)

Debt service (principal and interest payments on long-​ term debt) divided by operating and nonoperating expenses plus principal

Statement of revenues, expenses, and changes in fund net assets (Principal is found in the notes.)

Debt service ratio

For business activities, a ratio of 1.0 or higher indicates the service is self-​ supporting. Because governmental activities may rely in part on tax revenues, the fee might not be expected to cover all costs. The ratio of fees to expenses should line up with local policy on cost recovery. A high ratio threatens the availability of resources for service delivery. Ratios above 20 percent may be particularly risky for enterprise funds relying on ratings from bond agencies.

Notes: Principal must be added for debt service as it is not included in expenses for accrual accounting. Principal can be found in the notes to financial statements. Source: Adapted from William C. Rivenbark, Dale J. Roenigk, and Gregory S. Allison, “Communicating Financial Condition to Elected Officials in Local Government,” Popular Government 75, no. 1 (2009): 6–​7. Used by permission.

f in a n c i al c o n di t i o n a n aly s i s

Selfsufficiency

393

newgenrtpdf

Description

Financial indicator

Governmental activities

Liquidity

Calculation

Data source

Calculation

Data source

Ability to meet short-​term obligations

Quick ratio

Cash and investments divided by current liabilities (not including deferred revenue)

Statement of net assets

Cash and investments divided by current liabilities (not including deferred revenue)

Statement of net assets—​ proprietary funds

Solvency

Ability to meet long-​term obligations

Net assets ratio

Unrestricted net assets divided by total liabilities

Statement of net assets

Unrestricted net assets divided by total liabilities

Leverage

Degree to which total assets are financed with long-​ term debt Condition of capital assets as defined by remaining useful life

Debt-​to-​ assets ratio

Long-​term debt divided by total assets

Statement of net assets

Long-​term debt divided by total assets

Statement of net assets—​ proprietary funds Statement of net assets—​ proprietary funds

Capital assets condition ratio

1.0 − (accumulated depreciation divided by capital assets being depreciated)

Statement of net assets or notes to financial statements

1.0 − (accumulated depreciation divided by capital assets being depreciated)

Capital

Business activities (enterprise funds)

Statement of net assets or notes to financial statements

Interpretation

A high ratio indicates substantial ability to meet short-​term obligations. For business activities a ratio of about 2.0 is generally seen as reasonable. For governmental activities, a higher ratio is desired in order to accommodate the uneven receipt of general government revenues. A high ratio indicates a greater ability to meet longer-​term obligations. A high ratio indicates over-​ reliance on debt for financing assets. This may be particularly problematic for business activities relying on bond agency ratings. A high ratio indicates substantial investment in capital assets while a low ratio suggests that capital is not being replaced as quickly as it is depreciating. A ratio between 0.40 and 0.60 may be an appropriate target.

Source: Adapted from William C. Rivenbark, Dale J. Roenigk, and Gregory S. Allison, “Communicating Financial Condition to Elected Officials in Local Government,” Popular Government 75, no. 1 (2009): 8–​9. Used by permission.

other a naly t ic options

Financial dimension

394

Table 30.4  Financial Stock Indicators for Governmental Activities and Business Activities

f in a n c i al c o n di t i o n a n aly s i s

assessment. Because I already was familiar with our reports, I knocked out the Hutchinson part in about half a day.” Chaz proceeded to tell Dan that he also had gathered five years of data for the cities of Benson, Davis, and Gleason—​the governments Hutchison most often used for comparisons. “That took a bit longer,” he said, “but it wasn’t too much. I put the data into a simple one-​page dashboard showing our numbers compared to the average of our peers (Figure 30.1). I think you are going to like this.” Dan examined Chaz’s dashboard. At first glance, he thought the display might be too simple to even begin to express the city’s financial condition—​especially when compared to the 150-​page financial statements city officials had always relied on. However, he soon began to see in this simple dashboard a compelling story of Hutchison’s financial condition emerging from the comparisons of the city’s data over time and against its peers. “We covered our expenses with the revenues received each year—​so no deficits,” he observed. “Our charges-​for-​expense ratio in governmental activities is greater than the ratios of our peers, which council will like as it encourages reliance on user fee revenue. I also see our debt-​service and debt-​to-​asset ratios look better than those of our peers. Council members will like that, as they prefer to keep debt down and we do that more than our comparison cities.” “Yes, I saw those strong points, too” responded Chaz. “In fact, we had the lowest debt-​service and debt-​to-​assets ratios among the four cities, and the lowest net assets compared to our liabilities for solvency. Our solvency has been dropping a bit, but that is true for all our peers as well. What really caught my eye was our capital assets condition ratio.” “Help me make sure I understand that one” replied Dan. “This is saying that for our assets that can be depreciated, like buildings, streets, and vehicles, our capital assets ratio has dropped to about 0.25. So, across all those assets we are down to only one-​fourth of the useful life left?” asked Dan. “That’s exactly right!” Chaz answered emphatically. “I checked further, and the ratio is even lower for our streets. This really lines up with the warnings we’ve received from our streets and fleet directors, who have been saying we aren’t investing enough in our infrastructure and vehicles. The ratio for our buildings is pretty good at 0.46, indicating that they are about half depreciated. But we need to pay attention to needed capital replacement in streets and fleet, which this ratio analysis really makes evident.”

395

396

other a naly t ic options Assessment of financial condition at the government-wide level Key:

Hutchison

Peer average (Benson, Davis, and Gleason)

Governmental activities RESOURCE FLOW Interperiod equity Total margin ratio

Financial performance Percentage change in net assets

1.30 1.20 1.10 1.00 0.90 0.80 10% 5% 0% –5% –10%

Self-sufficiency Charge-to-expense ratio

Hutchison = 3.9%, Benchmark = 2017 2018 2019 2020 2021 6.9%

10%

2017 2018 2019 2020 2021

5%

2017 2018 2019 2020 2021

Quick ratio

4

Hutchison = 6.96, Benchmark = 6.05

Liquidity measures a government’s ability to meet its short-term obligations. The quick ratio—cash and investments divided by current liabilities—is used to analyze this dimension of resource stock.

2017 2018 2019 2020 2021

Hutchison = 0.28, Benchmark = 0.39

Solvency measures a government’s ability to meet long-term obligations. The net assets ratio is calculated as unrestricted net assets divided by total liabilities.

0.75 0.50 0.00

2017 2018 2019 2020 2021

0.20 0.10 0.00 2017 2018 2019 2020 2021

Capital Capital-assets condition ratio

Financing obligation looks at service flexibility by determining the amount of total expenses committed to annual debt service. The debt service ratio is calculated as annual debt service divided by total expenses.

1.00

0.25

Leverage Debt-to-assets ratio

Hutchison = 0.02, Benchmark = 0.05

6

0

Net assets ratio

Self-sufficiency addresses the extent to which service charges covered total expenses. The charge-to-expense ratio is calculated as charges for services divided by total expenses.

8

2

Solvency

Hutchison = 15.6%, Benchmark = 10.3%

10%

0%

RESOURCE STOCK Liquidity

Financial performance shows how much a government’s financial position improved or deteriorated as a result of resource flow. The percentage change in net assets is calculated as the change in net assets divided by beginning net assets.

20%

0%

Financing obligation Debt service ratio

lnterperiod equity measures whether a local government lived within its financial means. Hutchison = The total margin ratio—total financial 1.04, resources divided by total financial Benchmark = obligations—is used to analyze this 1.13 2017 2018 2019 2020 2021 dimension of resource flow.

Hutchison = 0.04, Benchmark = 0.10

Leverage measures how much total assets are financed with long-term debt. The debtto-assets ratio is calculated as long-term debt divided by total assets.

0.60

Capital is the condition of capital assets as defined by their remaining useful life. The Hutchison = capital-assets condition ratio is calculated 0.20 0.25, as accumulated depreciation divided by Benchmark = 0.00 capital assets being depreciated. This result 0.49 2017 2018 2019 2020 2021 is then subtracted from one. 0.40

Figure 30.1  Financial Condition Dashboard for Hutchison Governmental Activities

f in a n c i al c o n di t i o n a n aly s i s

“Lily is going to be concerned about some of these findings, but she is really going to like this model and dashboard,” commented Dan. “She’s been saying we need to address capital needs, but the council has been cautious. These ratios clarify the problem. Additionally, because we are so much lower in our debt service and debt leverage than the comparison cities, we might consider financing some of this with bonds given the current low interest rates. Even with some added debt, we’d still compare favorably with our peers. Anything else you see?” Dan asked Chaz. “I don’t see a lot of other issues, really. Other aspects of our financial condition look pretty good. I think the comparisons may help the council understand that,” offered Chaz. “I like that, and I think Lily will too,” Dan responded. “Let’s get on Lily’s schedule sometime before the week is up. This is like getting your medical test results back and seeing where you have some challenges but also where your health is OK. I think the council will find this information to be a lot clearer and easier to absorb than what we have given them in the past. How soon will you be ready to roll this out?” Chaz said he wanted to develop dashboards for the general fund and for the water and sewer fund to see what issues arose there. He also thought he should dig further into the capital condition data so they could determine where depreciation was greatest. Finally, he thought they might begin framing a capital budget for the next 5 or 10 years.

The value of measuring financial condition Failure to routinely perform an assessment of financial condition is like skipping one’s regular visits to the doctor and the tests that come with those visits. You might be OK for a while, but problems can be missed that really should be addressed. Waiting until the problems become serious can make solutions more difficult and painful. The North Carolina model ties the analysis of a local government’s financial condition directly to its annual financial statements. This linkage makes it easier to communicate the results to the governing body and affirms the importance of the financial reporting process. It also turns the financial data into a story that can help shape decision making. For the fictional city of Hutchison, some of the financial news was good—​or at least acceptable—​while other financial ratios indicated problems. This is likely to

397

398

other a nalytic options

be the pattern for all but the most affluent and the most financially stressed local governments. There will be strengths and weaknesses revealed in the assessment. Some of the problems an analysis reveals were probably not totally unknown by a local government’s managers before using the model, but the analysis can elevate these problems to the attention of senior managers and elected officials as they strive to fulfill their responsibility for ensuring the government’s financial sustainability.

References Brown, K.W. “The 10-​Point Test of Financial Condition: Toward an Easy-​To-​ Use Assessment Tool for Smaller Cities.” Government Finance Review (December 1993): 21–​26. Chaney, B.A., D.M. Mead, and K.R. Schermann. “The New Governmental Financial Reporting Model.” Journal of Government Financial Management 51, no. 1 (Spring 2002): 26–​31. Governmental Accounting Standards Board. “Statement No. 34.” In Basic Financial Statements—​ and Management’s Discussion and Analysis—​ for State and Local Governments. Norwalk, CT: Governmental Accounting Standards Board, 1999. Groves, S.M., W.M. Godsey, and M.A. Shulman. “Financial Indicators for Local Government.” Public Budgeting & Finance 1, no. 2 (Summer 1981): 5–​19. Ives, Martin, and Kirk Schanzenbach. Financial Condition Analysis and Management. Fort Worth, TX: Sheshunoff Information Services, 2001. Mead, Dean Michael. An Analyst’s Guide to Government Financial Statements. Norwalk, CT: Governmental Accounting Standards Board, 2001. Mead, Dean Michael. “Assessing the Financial Condition of Public School Districts.” In Selected Papers in School Finance, 2000–​01, edited by W.J. Fowler, 55–​76. Washington, DC: US Department of Education, National Center for Education Statistics, 2001. Nollenberger, Karl, Sanford M. Groves, and Maureen Godsey Valente. Evaluating Financial Condition: A Handbook for Local Government, 4th ed. Washington, DC: International City/​County Management Association, 2003. Rivenbark, William C., Dale J. Roenigk, and Gregory S. Allison. “Communicating Financial Condition to Elected Officials in Local Government,” Popular Government 75, no. 1 (2009): 4–​13.

f in a n c i al c o n di t i o n a n aly s i s

Rivenbark, William C., Gregory S. Allison, and Dale J. Roenigk. “Conceptualizing Financial Condition in Local Government.” Journal of Public Budgeting, Accounting, and Financial Management 22, no. 2 (2011): 241–​267.

Suggested for further information Mead, Dean Michael. “A Manageable System of Economic Condition Analysis for Governments.” In Public Financial Management, edited by Howard A. Frank, 383–​419. New York: Taylor & Francis, 2006. Rivenbark, William C., and Dale J. Roenigk. “Implementation of Financial Condition in Local Government.” Public Administration Quarterly 35, no. 2 (2011): 241–​267. Wang, Xiaohu. Financial Management in the Public Sector: Tools, Applications and Cases. Armonk, NY: M.E. Sharpe, 2006.

Web resources Kavanagh, Shayne, “Diagnosing the Financial Condition of Your Local Government” www.gfoa.org/ ​ m aterials/ ​ d iagnosing- ​ t he- ​ f inancial-​ condition-​of-​your-​local-​government North Carolina State Treasurer, “County and Municipal Fiscal Analysis” https://​ lgreports.nctreasurer.com/​lgcreport/​ Office of the New York State Comptroller, “Financial Condition Analysis: Local Government Management Guide” www.osc.state.ny.us/​files/​local-​ government/​publications/​pdf/​financialconditionanalysis.pdf Rivenbark, William C., Dale J. Roenigk, and Gregory S. Allison, “Communicating Financial Condition to Elected Officials in Local Government” www.sog. unc.edu/​sites/​www.sog.unc.edu/​files/​articles/​article1finance.pdf Also, consider a web search using these key words: local government financial condition, financial ratio analysis, and financial stress in local government. For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

399

31 RANDOM CHANCE OR BIAS? A PRACTICAL USE OF BINOMIAL PROBABILITY DISTRIBUTION

Sometimes it’s the mayor and sometimes the governing board who appoints citizens to serve on official citizen boards and commissions to address civic matters of one type or another. These citizen appointees are given a special opportunity to influence local affairs. It should not be surprising when some taxpayers observe the composition of citizen boards and commissions and wonder whether their own gender or racial group is being appropriately represented. The same concern may apply to the composition of the government’s workforce. After all, the local government is usually among the most prominent organizations and biggest employers in most communities. Are women, African-​ Americans, and Latinos fairly represented or under-​represented on citizen boards and in city or county departments? Is the composition of these boards and the workforce generally consistent with the proportions found in the community’s population? Fortunately, something called the binomial probability distribution gives local government managers and analysts an easy way to analyze the numbers and gauge the probability that their government’s current proportion of women

u s e of binom i al p ro babi l i t y di s t ri bu t i o n

or African-​Americans or Latinos serving on citizen boards or in the workforce would occur if there is no discrimination in citizen appointments or hiring practices.

Scenario: Wattlington, Kansas When Celine Mayer ran for the office of mayor, she promised economic development, good city services, and safe neighborhoods. She also criticized the incumbent mayor and all the “good old boys” on the city council and upper echelons of municipal administration for lack of inclusiveness in city government. The voters narrowly elected Celine, but they also returned to office most of the city council members she had labeled “good old boys.” The city of Wattlington operates under the council-​manager form of government, so the mayor presides over the city council’s meetings and can influence the direction that the city takes, but she is not the government’s chief executive. In Wattlington, the chief executive is city manager Arnie Seilheimer, who handles day-​to-​day management of the government, offers policy recommendations, and reports to the mayor and city council. Celine isn’t quite sure whether Seilheimer will be a problem or an ally in her quest for a more inclusive city government. At her very first city council meeting as Wattlington’s top elected official, Mayor Mayer chose not to raise the topic of inclusiveness. Instead, she focused on building bridges to members of the council and dealing with a few immediate matters facing the government. But at her second meeting, she introduced the subject of inclusiveness in city government and persuaded a majority of the council to ask the city manager to look into the question of how well women, African-​Americans, and Latinos were represented on citizen boards and in the ranks of city employees. Seilheimer promised to examine the matter and provide a report.

Representation in appointments and employment? The next morning, Seilheimer called management analyst Gerry Welch into his office. “I need for you to pull together statistics on the gender and racial composition of citizen boards and the city workforce. The mayor wants to know if their composition is out of sync with the city’s demographic

401

402

other a nalytic options

profile. It’s a good question. I am especially interested in the part we most have responsibility for—​the city workforce composition.” “Sure, I will get on this right away,” Gerry said. “Do you want me to look at each citizen board individually or should I combine them and look at citizen appointments as a whole?” “Let’s combine them and see how that looks.” “And, Gerry, I am not just asking for a table that displays the composition,” Seilheimer added. “I want some statistical analysis that tells us if we have a problem—​and if so, how serious.” “I’m on it,” Gerry said, exiting the office.

Binomial probabilities With fairly simple calculations (even simpler if one uses Excel!), the binomial probability distribution offers a way to estimate the probability of various outcomes—​for instance, the probability of tossing a coin five times and getting four heads and one tail or the probability of counting the people in a city park and finding that women outnumber men 82 to 21—​as long as a few common conditions apply. The first required condition is that the variable of interest must be dichotomous; that is, either it is heads or it is tails, male or female, a member of a particular racial group or not a member of that group. The second condition is that the probability of a result for any single instance is not affected by previous instances—​that is, the probability of getting heads on the next toss is not affected by the results of previous tosses. In other words, the probability is constant and independent of previous trials.1 The third required condition is that there is a fixed number of trials. You have to know, for instance, how many times the coin will be flipped. If these three conditions are met, then the binomial probability distribution can be used to determine the probability of, say, getting 56 or more heads in 100 tosses, or having only 12 women or only 4 Latinos appointed to the city’s citizen boards. To perform the necessary calculations, we must have three bits of information. First, we need to know how many times the coin was tossed or how many people were appointed to the citizen boards. As a statistician would 1 Statisticians refer to any process that meets these two conditions as a Bernoulli process. The binomial probability distribution is applicable only to these processes.

us e of b in om ial p ro bab i l i ty d i s t ri but i o n

say, we need to know the number of trials. Second, we need to know how many heads occurred or how many women serve on the boards or how many Latinos—​in statisticians’ terms, the number of successes. Third, we must know the probability of the event in question if the trial is unbiased—​ that is, the probability of tossing a head, if the coin is unbiased, or the probability of choosing a woman or a Latino in a blind selection. The formula for calculating the probability of a given outcome is provided in Box 31.1. The easier method of using Excel to do the calculations is explained in Box 31.2. In either case we usually are interested in knowing the probability of a specified range of “successes” (usually, a given number or more, or a given number or fewer) rather than the probability of a particular number on the button.

BOX 31.1  CALCULATIONS FOR USING THE BINOMIAL PROBABILITY DISTRIBUTION Tossing a coin 100 times could yield many different combinations of heads and tails. Similarly, appointing 100 citizens to boards could yield many different combinations of women and men, as well as many different combination of Latinos and non-​Latinos. To calculate by formula the probability of any particular result in a binomial distribution, we first have to know how many possible combinations of results there are. This requires us to use factorials in the equation. While the formula is explained here for those who want to know the math, the calculation more easily can be done using a scientific calculator or even more simply in Excel without the user even knowing the formula below. To perform the calculations, we need to know how many combinations (designated as “C”) are possible, which is like asking how many different combinations of heads and tails we might get in 100 coin tosses or how many combinations of Latinos and non-​Latinos, if the processes are truly random. The binomial probability formula is b(x; n,P) = nCx × P x × (1 – P)n–x where b = binomial probability x = number of successes (heads, female, Latinos)

403

404

ot h e r an alyt ic options

P = probability of success on an individual trial n = number of trials C = number of combinations given n and x, which expressed as factorials is: n x nCx

=

n! x! ( n − x ) !

In most cases the number of possible combinations is surprisingly high. Why so? Consider that there are 120 different combinations of getting seven heads in 10 coin flips (10C7 = 120). You could get seven heads followed by three tails; a head on the first flip, three tails, then six heads; a tail on the first flip, followed by seven heads and two tails; and so forth for 120 different combinations. The probability of getting seven heads on 10 tosses of a coin, given a 50 percent probability of heads on any single flip would be:

b = 120 × 0.57 × (1 − 0.5)

10-7

= 120 .0078125 × .125 = 0.1172

So the probability of getting exactly seven heads—​no more and no less—​ in ten flips of the coin would be 11.72 percent. If you want to figure out the probability of getting seven or more heads you will need to do the calculations for each of these outcomes (7 heads, 8 heads, 9 heads, and 10 heads) and then add their probabilities. In this case the probability of getting seven or more heads would come to 17.19 percent.

FROM THE ELECTRONIC TOOLKIT BOX 31.2  CALCULATIONS FOR USING THE BINOMIAL PROBABILITY DISTRIBUTION WITH EXCEL Calculating the binomial probability with Excel is much easier than doing it manually or with a calculator. Just use the BINOM.DIST function. This function requires four arguments or elements: =BINOM.DIST(Successes, Trials, Probability of Success, Cumulative).

u se o f b in om ial p r obab i l i t y d i s t ri bu t i o n

Figure 31.A  Binomial Distribution Function for Point Estimates in Excel Suppose that you want to know the probability of getting seven heads (successes) in 10 coin flips (trials), when you know that the probability of getting a head on any single flip is 50 percent (probability of success). We have already defined the first three elements. The last element is whether you want the calculation to be a cumulative calculation (TRUE means everything up to and including the number of successes) or if you want it to be a point estimate (FALSE means only exactly the number of successes). In Excel, the result would look like this for FALSE—​that is, the point estimate. There is a 11.72 percent chance of getting exactly seven heads after 10 coin tosses. If we instead wanted to calculate the probability for seven or more heads, we could find the cumulative probability of getting zero to six heads (less than seven) and subtract this from 1 to arrive at the probability of getting seven heads or more. Doing so tells us that there is a 17.19 percent chance of getting seven or more heads with 10 tosses of the coin.

Figure 31.B  Binomial Distribution Function for Cumulative Estimates in Excel

405

406

other a nalytic options

When the number of trials is small—​for instance, if the total number of citizens serving on citizen boards was only 20 persons—​it is not too difficult to do the calculations for using the binomial probability distribution. But with a large number of trials, the calculation becomes quite tedious. In such cases, the binomial probability distribution can be estimated using the normal distribution. To use the normal curve, we need to know the number of trials and the proportion of the variable of interest (e.g., female) in the general population. This will allow us to calculate the standard deviation of a probability distribution for a binomial probability estimate range:

σ = np (1 − p ) where n = the total number of persons serving on citizen boards p = the percentage of females in the adult population of the community (expressed in decimals—​for example, 52 percent = 0.52) Once the standard deviation is determined, it is a simple matter to convert the number of women actually serving on citizen boards into something called a z score and then use this z score and the normal distribution table (Appendix E) to find the probability that this number of women or fewer would have been appointed absent discrimination. The procedure for doing this will be explained in the case example below.

Testing the evidence of discrimination Gerry Welch asked the city clerk to compile information on the gender and racial composition of all the official citizen boards serving Wattlington, excluding the city council. He asked the human resource director to do the same for the city’s workforce of full-​time employees. Seasonal and part-​ time employees would not be included in the analysis. While Gerry awaited the information about citizen boards and the city’s workforce, he searched for and found information from the census bureau regarding the gender and racial composition of Wattlington’s adult population—​those 18 years of age and older. He decided that this, rather

u s e of binom i al p ro babi l i t y di s t ri bu t i o n

than the entire population including children, was the relevant pool for his analysis of citizen boards and city employment. Gerry learned that the adult population of Wattlington was: • •

48 percent male 52 percent female

The racial composition was: • • • •

44 percent white 33 percent African-​American 17 percent Latino 6 percent other

When the requested information arrived from the city clerk and the human resource director, it provided the needed details about citizen boards and full-​time employees, as shown in Tables 31.1 and 31.2. Gerry was struck immediately by an apparent imbalance favoring white males on both tables. He set out to do his analysis of the numbers, beginning with the citizen boards. The normal curve can be used to determine the probability that the gender split among Wattlington’s citizen board members could have Table 31.1  Gender and Racial Composition of Wattlington’s Citizen Boards

Male Female Total

White

Black

Latino

Other

Total

30 15 45

18 10 28

4 3 7

1 1 2

53 29 82

Table 31.2  Gender and Racial Composition of Wattlington’s Full-​Time Employees

Male Female Total

White

Black

Latino

Other

Total

171 98 269

53 81 134

49 45 94

8 15 23

281 239 520

407

408

other a naly t ic options

happened as it did if there were no gender discrimination in appointments. The following information is relevant to this analysis: p = .52, the probability of randomly selecting a female n = 82, the number of persons serving on citizen boards X = 29, the number of females serving on citizen boards μ = the number who would be female if no discrimination was in evidence (n × p). In a group of 82 board members, we would expect 42.64 to be women (82 × 0.52 = 42.64). The first two of these numbers make it possible to calculate the standard deviation of the population. σ = np (1 − p ) In this case, Gerry performs the following calculation: σ = np (1 − p ) = 82 × .52 (1 − .52 ) = 82 × .52 × .48 = 20.47 = 4.52 Knowing the standard deviation (σ = 4.52), Gerry converts the number of women actually serving on citizen boards (29) into a z score, telling us how many standard deviations this number is from the population mean. (x − µ) σ 29 − 42.64 = 4.52 = −3.02 (or 3.02 standard deviations below th he mean )

z=

At three standard deviations below the mean, this score is almost off the chart. It is a very abnormal result, if there is actually no gender

u s e of binom i al p ro babi l i t y di s t ri bu t i o n

discrimination. We know this because we know that 68.26 percent of all values in a normal distribution lie within one standard deviation of the mean (so more than two-​thirds of all values would all have a z score between +1.0 and −1.0); 95.44 percent of all values lie within two standard deviations of the mean (z scores between +2.0 and −2.0); and 99.72 percent of all values lie within three standard deviations of the mean (z scores between +3.0 and −3.0). At −3.02, we know this z score lies slightly outside even this group. To find the probability that the gender makeup of citizen boards would be so out of sync with the gender proportions of the general population, Gerry looked up the z score in the normal distribution table (see Appendix E). This table converts z scores into probabilities. The numbers down the left column of the table are associated with the first digits of a z score and the numbers across the top row are associated with the final digit of the z score. In Gerry’s case, the z score is −3.02, so he proceeds all the way down to 3.0. Next, he checks the top row to find the number corresponding with the z score’s final digit. A few columns from the left margin, Gerry locates 0.02, which completes his z score of 3.02. (The probability of −3.02 standard deviations from the mean is the same as the probability of +3.02 standard deviations from the mean, so the probability revealed here will work in either case. A negative z score refers to standard deviations below the mean—​i.e., the left portion of the normal curve—​and a positive z score refers to standard deviations above the mean—​i.e., the right portion of the normal curve.) The 3.0 found down the left column and the 0.02 in the top row intersect at 0.9987, which means that 99.87 percent of the area under the normal curve lies between the point 3.02 standard deviations below the mean (−3.02) and the tip of the opposite tail. This tells Gerry that the probability of getting 29 or fewer women appointed to Wattlington’s citizen boards, if there is no gender discrimination, is 0.0013 or only 0.13 percent—​barely more than one-​tenth of 1 percent. Slim odds, indeed! Gerry followed a similar procedure in analyzing the racial composition of persons serving on citizen boards, but with a twist. Remember, one of the required conditions for binomial analysis is that the variable of interest must be dichotomous. Because no one in the gender analysis had identified as something other than male or female, binomial analysis worked fine—​all

409

410

ot h e r an alyt ic options

citizen board members identified as either male or female.2 For the racial composition analysis, Gerry needed to examine racial groups one at a time to create dichotomous variables—​that is, white and non-​white (combining African-​American, Latino, and other) for one analysis, African-​American and non-​African-​American for another analysis, and Latino and non-​Latino for yet another.

BOX 31.3  WHEN ARE A STUDY’S FINDINGS SIGNIFICANT? As noted earlier in this book, the thresholds of statistical significance used by local government analysts and managers may differ from those used by scholars in research appearing in academic journals (see Box 9.1). When a statistical result is found to have a probability of 5 percent or less (p < .05), this means that there is only one chance in 20 that the findings in that study could have happened by chance. This is a strong enough indication that underlying factors are influencing the result for scholars and academic journals to regard this to be a statistically significant finding. Such a finding would be regarded significant in a study conducted by a local government as well, but so might findings having a less stringent threshold. A local government study examining a serious matter—​for instance, the disparate rates of injury associated with different manufacturers’ playground equipment in use in local parks or allegations of racial discrimination in the delivery of government services—​would, of course, produce a government response if the evidence of hazard or bias was 20 to 1 (p < .05). But the government probably should also respond if the study produces findings approaching but not reaching that threshold. In a local government environment, professional managers and elected officials will want to react if there is a 75 or 80 percent probability that something untoward is influencing an undesirable result. The significance threshold for management analyses in local government in most cases should be set at a more generous range than in academic studies. We offer these suggested ranges:

2 If someone had identified as something other than male or female, the analysis could have proceeded. Gerry could have combined categories to create dichotomous variables—​e.g., female and not female (a category combining male and other).

u s e of b inom i al p ro bab i l i t y d i s t ri bu t i o n

p < .25 Consider this to be a significant finding for practical administrative analyses. If there is a 75 percent probability that a result of concern is happening not by chance but because an underlying factor is influencing the result, some further investigation and possible corrective action may be warranted. .25 < p < .40 Consider a probability in this range to warrant discussion with advisers. Be cautious about asserting anything definitive here, because the evidence is not overwhelming. However, take into consideration the ramifications of mistakenly rejecting the hypothesized relationship of concern. Failing to act on findings even in this range could seem negligent in retrospect. p > .40 The results of this analysis do not warrant action. We also offer a pair of caveats. First, these rules of thumb apply to special analytic studies undertaken infrequently, not to the daily, weekly, or monthly analyses of routine operating statistics. For daily, weekly, or monthly analyses of routine operating statistics, which will produce a large number of trials over a one-​or multiyear period, it is more appropriate to use either the traditional academic threshold (p < .05) or the testing standards of control charts (see Chapter 27). Second, a finding that is statistically significant might still fail to be substantively significant. Sometimes a tiny difference that nevertheless achieves statistical significance may be regarded as too small to be worth the effort and expense necessary to close that small gap.

Gerry began the racial composition analysis by focusing on the appointment of white members of citizen boards compared to the appointment of members of all other racial groups combined.The following information is relevant to this analysis: p = .44, because 44 percent of Wattlington’s adult population is white, this is the probability of randomly selecting a white appointee for a citizen board n = 82, the number of persons serving on citizen boards X = 45, the number of white appointees serving on citizen boards μ= the expected number of white appointees if no discrimination was in evidence (n × p). In a group of 82 board members, we would expect 36.08 to be white (82 × 0.44 = 36.08).

411

412

other a naly t ic options

The first two of these numbers make it possible to calculate the standard deviation of the population.

σ = np (1 − p ) In this case, Gerry makes the following calculation:

σ = np (1 − p ) = 82 × 44 (1 − .44 ) = 82 × .44 × .56 = 20.20 = 4.49 Knowing the standard deviation (σ= 4.49), Gerry converts the number of white appointees actually serving on citizen boards (68) into a z score, telling us how many standard deviations this number is from the population mean. z=

(x − µ) σ

45 − 36.08 4.49 = 1.99 =

“Oh, man, that’s practically two standard deviations above the mean,” Gerry muttered to himself. “The gender split was already troublesome enough, but this won’t look good for the representativeness of our citizen boards along racial lines either.” Gerry went back to the normal distribution table to get the precise numbers. The 1.9 found down the left column and the 0.09 in the top row intersect at 0.9767, which means that 97.67 percent of the area under the normal curve lies between the point 1.99 standard deviations above the mean and the tip of the opposite tail. This tells Gerry that the probability of

us e of b in om ial p ro bab i l i ty d i s t ri but i o n

getting 45 or more white members without the presence of racial discrimination in appointments is only 2.33 percent. Gerry still needed to do the analysis for other racial groups on citizen boards and the gender and racial analysis of full-​time employees, but he would come back to that. Right now, he thought he should share his initial findings with city manager Seilheimer. Later on, when Gerry returns to complete the analysis, he will need to bear in mind an important limitation on the use of the normal distribution table. The normal curve is a good approximation of the binomial distribution when n × p is greater than 10 and n × (1 –​p) is also greater than 10, as they were in both analyses Gerry has performed so far. When he moves to analyses of the African-​American and Latino categories, he will again meet this threshold; however, when he gets to the “other” category, which in Wattlington constitutes a much smaller percentage of the population, he will find that n × p and n × (1 –​p) fall short of the required threshold. To analyze that category, he will need to use the binomial distribution rather than the normal curve. Gerry will also need to keep special considerations in mind when combining groups to analyze possible discriminatory effects disadvantaging one group versus all others (see Box 31.4).

BOX 31.4  A WORD OF CAUTION WHEN COMBINING GROUPS FOR ANALYSIS An instance in which one racial group is favored over all other groups presents the need for special handling by the analyst. The case described in this chapter is such an instance. Because white candidates for appointment and employment in the fictional city of Wattlington are successful in gaining posts at a rate disproportionate to their ratio of the general population, a statistical test that examines the appointment of whites versus non-​whites not surprisingly suggests the likelihood of discrimination in appointment and hiring practices. The probability that the disparity favoring whites could happen simply by chance was found to be quite low. If the analyst proceeds to perform subsequent analyses regarding the appointment and hiring of African-​Americans and Latinos, special

413

414

o the r analyt ic opt ion s

care may be needed. An analysis of the appointment to boards and the employment of African-​Americans versus non-​African-​Americans (combining whites and Latinos) could understate the level of discrimination against African-​Americans. If Latinos also are disadvantaged by discriminatory practices, their presence in the comparison group along with whites would blunt the statistical impact. A similar effect would occur in the analysis of appointment and employment of Latinos versus non-​ Latinos due to the presence of African-​Americans in the latter group. In cases where one large segment of the population is favored disproportionately over all others, an analysis revealing this particular discrimination may offer the most accurate depiction of reality—​more so than analyses attempting to depict discrimination against other individual groups versus all others.

Interpreting the results in Wattlington As usual, the city manager had a full schedule for the day, but his secretary Susie Watson suggested that Gerry drop back by right after lunch. She thought the manager might be able to squeeze him in for 10 or 15 minutes. Gerry arrived a few minutes early, waited, and got his time with Seilheimer. “I haven’t finished the analysis,” Gerry said, “but I wanted you to know that I am finding the composition of the citizen boards to be heavily skewed toward white citizens and away from women. The findings could make a pretty big splash, so I wanted you to know.” “Sounds like it confirms what the mayor was saying,” Seilheimer replied. “A lot of people won’t like this news, especially the incumbent council members who have been making these appointments. We’ll need to give some thought to whatever assistance we can provide, assuming they want to see changes in future appointments.” “What do you mean? How could we make a difference on this?” “Well, we might be able to do some things that would broaden the pool of citizens interested in serving on these boards. Then the council would have more options for their choices. Some cities hold citizen academies to teach interested citizens about city government and introduce them to the array of functions we perform. Many of the graduates go on to become

u s e of binom i al p ro babi l i t y di s t ri bu t i o n

engaged on advisory boards. I suspect there are other ways we might help, as well.” “I will give this some thought and make a few phone calls to see what other cities are trying,” Gerry said. “What about your analysis of gender and racial patterns in city employment?” Seilheimer asked. “That’s the one that is fully in our court rather than the council’s.” “I haven’t tackled that one, yet,” Gerry replied. “When I found the big disparities on the citizen boards relative to the general population, I wanted to give you a heads up. I should have some numbers on the city workforce tomorrow.” “Ask Susie to block off a little time for us tomorrow afternoon.”

Suggested for further information City of Raleigh Boards and Commissions: A Demographic Study & Review of How Boards Are Managed. Raleigh, NC: City of Raleigh Strategic Plan Organizational Excellence Initiative 3.2 Team, January 2019. https://​ go.boarddocs.com/​nc/​raleigh/​Board.nsf/​files/​B96UP971D290/​$file/​ 20190212BMSBoardCommissions_​RptExcerpt.pdf

Web resources Glen, Stephanie. “Binomial Distribution: Formula, What It Is and How to Use It.” www.statisticshowto.com/​probability-​and-​statistics/​binomial-​ theorem/​binomial-​distribution-​formula/​ Valcheva, Silvia. “Examples of Binomial Distribution Problems and Solutions.” www.intellspot.com/​binomial-​distribution-​examples/​ Also, consider a web search using these key words: binomial distribution tests, binomial statistics. For a template and exercise associated with this chapter, see https://​ toolsfordecisionmaking.sog.unc.edu.

415

32 FORECASTING

In one of several memorable quotes attributed to baseball hall-​of-​famer Yogi Berra, the New York Yankee great supposedly said, “Prediction is hard, especially when it’s about the future.” He was correct, of course. Predicting the future is difficult. Yet trying to “crystal ball” the future is an everyday event for weather forecasters, stockbrokers, oddsmakers, political pundits, and, though a bit less frequently, many other occupations. Even some local government administrators have occasion to make forecasts as part of their professional duties. Getting their predictions right can be very important. Each year city and county finance officials are asked to predict revenues for the upcoming fiscal year, and most are remarkably accurate—​usually missing by no more than a few percentage points. Their success rarely is attributable to some mystical formula or sophisticated forecasting model or the blind application of a trend line of prior years. More often, their successful forecasts are based on awareness of what is happening in their community—​new neighborhoods being developed, new shopping malls or office buildings under construction, troubling signs of slowing production at a local industry—​and how these occurrences will affect taxes and other government revenues. Awareness of local conditions and developments informs their expert judgment, which they

f o re c a s t i ng

apply, as relevant, to projections for each revenue category before adding these projections together to arrive at a total for all revenues. Can statistical forecasting based on models or trend lines be helpful? Certainly. But as we will see in the case of the fictional city of Misty Glen, trend-​ line forecasting may be most useful for long-​term projections, where finance officials have fewer real-​time insights to guide their forecasts, than for short-​ term projections, where the most perceptive have many.

Scenario: Misty Glen, Pennsylvania Pam Beastley replaced Stanford Huddleston as finance director of the city of Misty Glen four months ago. Stan retired after a long career during which he acquired legendary status among municipal finance officials across the state. This is Pam’s first experience as the head of a municipal finance department, but she is well qualified by education and experience below the level of director. She inherited a good department at Misty Glen and the transition has gone smoothly. Today she is meeting with city manager Michael Scoggins and budget director Jim Haltom about next year’s budget—​her first as finance director. “Jim will take the lead in reviewing departmental budget requests,” Michael began, “but we always count on the finance director to forecast our revenues for the coming year. Year after year, Stan would give us a number that was always pretty close to the total revenues we actually would receive.” “Always close and usually on the low side,” Jim interjected. “He tried to give us a conservative estimate of revenues, so if he missed the mark a little, we would end up with a little more revenue than he had projected.” “We are counting on you to do the same,” Michael said. Pam assured Michael and Jim that she would give this her careful attention and would do her best to give them a revenue projection they could count on. As she walked back to her office, she thought about the approach she would take and whether it would need to be different than Stan’s approach to revenue forecasting. Stan had served as finance director for so many years and had become so familiar with all of Misty Glen’s revenue sources and how they reacted to various stimuli in the local environment that he had developed a wonderful sense of what to expect in the coming year. As far as Pam could tell, Stan’s forecasts were more the

417

418

other a nalytic options

product of this sense and a little arithmetic than they were the result of super-​sophisticated calculations. To replicate Stan’s approach, Pam knew she would need to learn as much as she could as quickly as she could about local developments. When she got back to her office, Pam put in calls to the planning director, economic development director, public works director, and utilities director. She began setting up meetings to learn the latest about development plans and even rumors that might be relevant to the local economy. She also made herself a note to visit with the executive director of the local chamber of commerce. Information from each of these contacts could influence her projections of city revenues. Pam called longtime assistant finance director Dwight Root into her office and told him she was counting on his help in making revenue forecasts. “Sure,” Dwight exclaimed, “I’d be glad to help. Just in the past couple of months I’ve learned how to use some spreadsheet trend-​line forecasting tools that I think will be great for this.”

Forecasting methods Local government officials—​not just finance officials but others as well—​ need to forecast a wide variety of numbers, some regularly and others only occasionally. These include revenues, of course, but also population, anticipated attendance at various programs and events, and calls for service, to name just a few. Methods for making these forecasts vary widely. One way to categorize forecasting methods is to divide them into qualitative techniques (including what the experts call naïve, judgmental, consensus, and expert forecasting as well as the use of the Delphi method1) and quantitative techniques (including deterministic techniques, trend analysis, and econometric forecasting).2 1 The Delphi method is an interactive forecasting technique in which a panel of geographically dispersed experts responds to a set of questions in two or more rounds. Between rounds the panel is provided a summary of the responses from the previous round. Through this technique the experts tend to move gradually toward consensus but are less likely than in a face-​to-​face panel meeting to be drawn to the position of an especially outspoken or dominant advocate of a particular position or forecast. 2 Barry Blom and Salomon A. Guajardo, Revenue Analysis and Forecasting (Chicago: Government Finance Officers Association, 2001).

f o re c a s t i ng

AVOID THIS COMMON ERROR BOX 32.1  MISINTERPRETING A STRAIGHT, ASCENDING TREND LINE AS DEPICTING A CONSTANT RATE OF GROWTH Does a straight line with an upward trajectory to the right depict a constant rate of growth? It seems that it might, but it does not! If the data produce a straight line in an ascending direction, the rate of growth actually is getting smaller and smaller. Compared to the base, which in this case would be growing every year, each year’s new growth represents a smaller and smaller percentage. If the data points remain arrayed in a nice, straight line, the trajectory is upward and growth may seem steady, but the rate of growth is declining. Growth at a constant rate would not be linear. Instead, when plotted against time the data would form a curved pattern appearing concave from above.

Another way to categorize forecasting techniques is to group them among four basic methods: • • • •

expert judgment deterministic techniques trend analysis econometric techniques

However, even with this variety of forecasting techniques available, experts caution that effective forecasting “is as much a professional art as it is a technical procedure.”3

Expert judgment Sometimes the forecaster’s insights on a given variable, perhaps from long experience and general familiarity with how various stimuli influence changes in that variable, provide a level of expertise that allows the 3 Douglas F. Morgan, Kent S. Robinson, Dennis Strachota, and James A. Hough, Budgeting for Local Governments and Communities (Armonk, NY: M.E. Sharpe, 2015), 195.

419

420

other a nalytic options

forecaster to project the variable’s future value with reasonable accuracy and without a lot of calculations. Expert judgment as the method of projecting local government revenues, for instance, relies on the forecaster’s knowledge of the local revenue system and insights regarding factors that influence growth or decline. Forecasts based on expert judgment can be less transparent than those derived by other means, so the forecaster should be prepared to explain the rationale for their projections. They must also recognize that broad trust in expert judgment can only be earned through a track record of accurate forecasts.

Deterministic techniques Some variables, including at least several categories of local government revenue, can be projected on the basis of simple calculations involving only a few factors (for example, assessed value × millage rate × collection rate = property tax revenues). If the forecaster has a reasonable estimate of each factor, projecting the variable is easy. Sometimes changes in these variables are less the product of overall trends than the result of specific changes in a few key factors. Consider, for instance, the forecasting of swimming pool admission revenues or revenues from library fines. The forecaster knows that pool admission revenues are the product of the number of admissions times admissions fees; however, the forecaster also knows that the recreation department plans to boost the fees by 25 percent next year and that the community’s recent growth in population—​especially families with young children—​is likely to give a special boost to attendance. Similarly, revenues from library fines might be based less on general trends than on planned increases in the levy for an overdue book and its anticipated deterrent effect on late returns. Deterministic techniques focus on key drivers of the variable being forecast—​factors such as predicted changes in tax rates, fees, population, service contracts, economic conditions, and the like. Factors thought to be relevant to the forecast are projected and built into the forecast calculations.

Trend analysis Trend or time-​series methods are easy to implement and are considered likely by some authorities “to be the backbone of most local government

f o re c a s ti ng

efforts at breaking away from their overreliance on judgmental techniques.”4 These methods extrapolate forecasts for the future on the basis of trends of the past and work best for forecasting variables that are rarely volatile. Revenue sources that have changed only gradually over the years, for example, are especially suitable for this method of forecasting. Time-​series forecasting techniques rest on the assumption that a given variable, perhaps revenues or expenditures, will be influenced by the same set of factors and in the same way as in previous years. A good analyst using these techniques must be alert, however, for variables or components of variables that have been subjected to actions or events that will cause sharp departures from the general trend—​for example, the effects of annexation or reappraisal on property tax revenues or the effects of a fee increase on service demand and revenues.

FROM THE ELECTRONIC TOOLKIT BOX 32.2  TREND ANALYSIS The actual application of trend analysis may be divided into four tiers, each with its own set of operations: elementary trend analysis, trend analysis using simple regression, trend analysis using simple regression with differencing, and exponential smoothing. Each may be performed with sufficient ease that some forecasters will choose to do all four before deciding on their forecast. Elementary trend analysis. Analysts using the most basic form of trend analysis to forecast revenues begin by simply calculating the year-​to-​year percentage growth or decline over a series of years. Suppose, for example, that during the past decade sales tax revenues had experienced annual growth rates as low as 1.1 percent and as high as 9.3 percent. Suppose, further, that the simple arithmetic mean (average) of these annual rates was 4.7 percent. Based on elementary trend analysis, the analyst would predict that sales tax revenues would grow by 4.7 percent in the coming

4 Howard A. Frank, Budgetary Forecasting in Local Government: New Tools and Techniques (Westport, CT: Quorum Books, 1993), 34.

421

422

other a naly t ic options

year and, if making a three-​year or five-​year forecast, by 4.7 percent annually on into these future years.1 An analyst wishing to make a conservative forecast might project a growth rate at the lowest rate of the prior decade’s range (1.1 percent) or at a rate midway between the lowest rate and the decade’s average (2.9 percent would be the midpoint).2 If the pattern of growth or decline is erratic and the analyst wishes to smooth the data for analysis, the data may be converted to moving averages prior to performing trend analysis (see Chapter 4). With a moving three-​year average, the range of averages in the sales tax example above would exhibit less variation; the low would be greater than the 1.1 percent single-​year low and the high would be less than the 9.3 percent single-​year high. The same tactic may be applied to trend analysis using simple regression, with or without differencing. Trend analysis using simple regression. Suppose, now, that we plot the past 10 years of sales tax revenue on a graph that has years marked along the horizontal X axis and revenues scaled up the vertical Y axis. Our data points on the graph show a general growth in revenues, but in this and in most cases they do not line up perfectly; in our example, no single straight line can be drawn through these data points perfectly. This is where regression analysis comes in handy. Linear regression is a technique that calculates the straight line that comes closest to the data points as a whole—​in other words, the line of best fit. (Regression is further described in Chapter 33 and Appendix F.) The basic formula is Y = a + bX where Y is the dependent variable (revenue, in this case) a is the Y intercept (where the regression line passes through the Y axis) b is the slope of the line X is the independent variable (the time periods) Statistical programs and even spreadsheets make linear regression easy to perform (see Appendix F for instructions). When the computer identifies the coordinates of the best fit line for years 1 through 10, it is a simple matter to extend the line to year 11 and, if desired, beyond to see the revenue forecasts for those years.

f o re c a s t i ng

As a shortcut method, simply use the Trend function in Excel as shown below. (This example shows the formula that may be entered in an empty cell to perform the desired calculation, when the observed periods are entered in cells A2 through A7, the observed data are entered in cells B2 through B7, and the succeeding period for which a forecast is desired is entered in cell A8.)

Figure 32.A  Excel Spreadsheet Displaying Trend Function to Calculate Forecast Trend Regression Line Analysts should take care in simply accepting the projection of the extended regression line, especially if the experience of recent years deviates sharply from earlier periods. Forecasting authorities contend that techniques such as exponential smoothing that give greater weight to more recent periods are more likely to produce accurate forecasts than techniques giving equal weight to all periods.3 Trend analysis using simple regression with differencing. Technically, regression analysis is intended for use with time-​series data only when the data for a given variable are not correlated from period to period. Unfortunately, our sales tax data violate the technique’s assumption on this score—​the presence of correlation becomes apparent when we consider how much our knowledge of last year’s sales tax revenue will help us predict next year’s revenue. Differencing is a tactic to reduce the correlation. Linear

423

424

other a naly t ic options

regression is performed once again, but this time the Y axis (dependent variable) is the change in sales tax revenue from the prior year (growth or decline) rather than sales tax revenue as a whole. Exponential smoothing. A trend analysis technique favored by some forecasters, exponential smoothing bases its prediction for the next period on the forecasts for previous periods and the weighted errors of those forecasts. Technically, the weighting—​ called the damping factor—​can range from 0 to 1, but a weighting between 0.2 and 0.4 is most common and usually works best. Because this technique is tied to previous forecasts and to the difference between the prediction and actual result, new projections tend to lag the pattern of actual results. A damping factor of 0.2 causes the forecast to adjust more gradually than a damping factor of 0.4. The Microsoft Excel spreadsheet program will calculate forecasts by the technique of exponential smoothing. To perform the calculation, enter the set or sets of data for the actual observations (for example, population, revenues, attendance, or fire incidents) in columns of the spreadsheet. Each column of data should be arrayed from the earliest period at the top to the most recent period at the bottom of the column. Be sure to leave a blank cell at the base of each column. (For directions on opening Excel and entering data refer to Box 1.1.) Next, go to the Data tab and click on “Data Analysis.” (If the Data Analysis package has not been installed, see Appendix F for installation instructions.) In the Data Analysis drop-​down box, select “Exponential Smoothing” and click “OK.” An “Exponential Smoothing” box will appear, requesting an input range, a damping factor, and an output range. The simplest way to enter the input range is to click the icon at the right end of the open “input range” rectangle, highlight the column of past observations on the spreadsheet plus the blank cell at the bottom of the column (the blank cell tells the computer that you want a forecast for the next period), click “enter,” enter a damping factor of your choice between 0.2 and 0.4 (0.3 is a reasonable “default” value), click the icon at the right end of the open “output range” rectangle, highlight a cell on the spreadsheet where you wish to have the results displayed, click “enter,” place a check mark in “chart output,” and click “OK.”

f o re ca s t i ng

Figure 32.B  Excel Spreadsheet with Exponential Smoothing Box Displayed for Forecasting Estimate

A new column of numbers and a line chart will appear. The column of numbers shows the exponential smoothing forecasts for the upcoming period (at the bottom of the column) and for earlier periods, too. “N/​A” appears at period 1 because no prior data were entered for the calculation of a first-​period forecast. The line chart compares forecasts to actual results. 1 Some local governments routinely extend their forecasts for three to five years into the future and some even make 10-​year forecasts as they prepare capital improvement plans. Not surprisingly, the accuracy of revenue forecasts declines the further into the future they extend. 2 The reason that many revenue forecasters prefer conservative estimates is that they hope to avoid the difficulties that come late in the year when on occasion revenues fall short of the amount projected. They much prefer the more pleasant year end when revenues come in greater than the conservative forecast. 3 Howard A. Frank, Budgetary Forecasting in Local Government: New Tools and Techniques (Westport, CT: Quorum Books, 1993), 73.

In instances such as these, the analyst may choose either to adjust the data points in the trend line to remove the effects of the unusual event or, alternatively, to exclude this particular category or subcategory from the trend line and its projection and handle this element separately. For example, if revenues from fees are based on a relatively stable element (growth in

425

426

other a nalytic options

licenses or demand for a given service, for example) as well as a volatile element (such as periodic adjustments in the amount being charged for the license or service), the analyst might use trend analysis to forecast the relatively stable element and make special adjustments based on separate predictions regarding the volatile element. In summary, trend analysis is a systematic forecasting technique; it produces a logical projection; and it has the advantage of simplicity and transparency. Its principal disadvantages lie in its assumption that effects of the past will persist into the future and in its inability to predict a turning point in the trend—​that is, an upturn in a generally downward pattern or a downturn in a generally upward pattern.

Econometric techniques Econometric techniques rely on statistical models developed by the analyst to project future values of a given variable. The models rely on the past relationship between a set of independent variables (for example, population, average personal income, average daily visitors) and the dependent variable (for example, sales tax revenue), and they forecast future values of the dependent variable on the basis of anticipated changes in the independent variables. Unlike trend analysis, which assumes that past trends will continue, econometric models invite the forecaster to insert into the calculation anticipated changes in the independent variables during the forecast period. Of course, accurately anticipating these changes in independent variables is a major challenge for the analyst. In contrast to trend analysis projections, these models can forecast turning points in upward or downward revenue trends based on anticipated changes in independent variables.

Assessing forecasting methods Which method is best? It is difficult to say. Although advanced statistical methods are appealing for their apparent objectivity, their ability to be refined over time, and their usefulness even following the retirement or departure of forecasting experts, advanced methods do not always produce forecasts more accurate than less sophisticated methods. Qualitative methods usually impose fewer data collection requirements than quantitative methods and produce forecasts that more quickly incorporate changing conditions.

f o re c a s t i ng

Judgmental and trend techniques are believed to be the most common forecasting methods of local governments although it is difficult to claim that they are always the best. In some cases it may be most appropriate to disaggregate elements of the forecast—​for instance, dividing revenues into individual types—​and to use different forecasting techniques for various categories, applying whichever technique is most suitable for a given category.

Scenario: Back in Misty Glen Two days later, Dwight excitedly placed a set of graphs on Pam’s desk for her to examine. “I’ve used trend analysis software on my spreadsheet program to project revenues for next year and two more years into the future, based on our actual revenues for the last six years,” Dwight declared (see Table 32.1). “I think we should project next year’s property tax revenue at that level and sales tax revenue at that level,” he said, pointing to next year’s spots (year 7) on the dashed lines on two of his graphs (Figure 32.1). Both lines extended beyond year 7 to include years 8 and 9 as well. “Thanks, Dwight,” Pam said, studying the graph. “Clearly you’ve been busy.” She paused before continuing, “Let me ask a couple of questions. Does this projection take into consideration tax revenues for next year associated with the current project expanding the West End Mall? And the rapid development of the neighborhood around the new Kerwyn Malone Elementary School?” Dwight hesitated, then answered, “Well, no.” Stammering a bit, he recovered to say, “At least, not exactly. The projection assumes the same level of growth Table 32.1  Misty Glen Revenues: The Six Most Recent Years

Six years ago Five years ago Four years ago Three years ago Two years ago Last year

Property tax ($)

Sales tax ($)

Occupancy tax ($)

Other tax & nontax revenues ($)

Total revenues ($)

16,342,318 18,690,329 21,276,968 21,713,205 23,081,521 25,267,247

5,879,696 6,424,167 6,974,456 7,654,959 8,456,040 9,138,060

636,233 589,942 563,486 572,932 675,295 783,509

470,742 489,012 436,559 489,122 569,733 582,638

23,328,989 26,193,450 29,251,469 30,430,218 32,782,589 35,771,454

427

428

other a naly t ic options

Figure 32.1  Forecasting Revenues for the Upcoming Year by Linear Regression Trend Line

in the tax base as we’ve had in the past few years. So it depends on whether those developments and others keep us in line with growth in the past.” Pam thought about Dwight’s response for a minute, before asking, “What if the things we know about right now—​some positive and some negative—​suggest that next year will be different from last year and the year before? Should we ignore that and just trust the trend line?” “I suppose you could make some adjustments to the trend line, based upon what you know,” Dwight replied. Pam was silent again, then asked, “How reliable are these projections, Dwight? If Stan Huddleston had used your trend line software to make his revenue projections prior to the last three years, would his projections have been better than the ones he actually made? Why don’t you test it and see? Pull the data from the years leading up to those projections and, pretending that you don’t know how things actually turned out, make your projections for the past three years. Then compare your results and Stan’s projections to the actual revenues.” Dwight left the office optimistic that his computer-​generated projections would fare well, even knowing Stan’s reputation as a good forecaster of revenue. Three days later he returned, but with a little less bounce in his step. “As you know, we have four major categories of revenue: property tax, sales tax, occupancy tax, and a catch-​all category of ‘other’ revenues,” Dwight began. “I did a separate projection for each category for each of the three years, so that’s 12 projections. Of those 12, Stan’s projections were closer to the actual revenue in nine cases. Mine were closer in three cases.”

f o re c a s t i ng

“I see,” Pam said, shaking her head slightly. Seeing her reaction, Dwight blurted, “But do you really think you can match Stan’s projections?” He quickly realized his faux pas and tried to recover. “Sorry, I didn’t mean that to sound as smart-​alecky as it did.” “That’s OK,” Pam replied. “I’ve been doing my homework and the other department heads have really helped bring me up to speed on what’s going on in the community and the effects on our tax base. Actually, I’m feeling pretty good about my projections.” She continued, “I am pleased that you’ve developed skills in using spreadsheet projections, but I’m not sure I should rely on trend lines entirely. However, I have two things in mind where your projections will be helpful. First, I would like to compare your projections for next year to mine, just as a check and to see if our projections are close and to consider whether I want to make any adjustments in mine. Second, I know that the city manager is doing some initial work preparing for a 10-​year strategic plan. He’s going to want some long-​range revenue forecasts for that 10-​year period—​ forecasts that will rely much more on broad assumptions about the future than on current plans or current rumors. We’re going to need those trend line projections to show what various assumptions will mean for future revenues. We can construct different scenarios and trend lines based on different assumptions. I said ‘we’ but I really mean, you can construct those trend lines.” “Actually, it’s pretty easy,” Dwight replied, now feeling much better about his new skill. “I’ll show you how to do it.” When Dwight left her office, Pam reviewed several approaches she was considering for making her projections: •

Previous rates of change. Pam could examine revenues and rates of change in the past and base her projection of next year’s revenue on recent history or the history over several years. Among her options were: • use last year’s rate of change to project next year’s revenue—​or perhaps use the average rate of change of the past two years • apply the mean or median of several years’ rates of change to this year’s revenue • for a conservative projection, apply the smallest rate of increase during the past several years (or the largest rate of decline) to this year’s revenue

429

430

other a naly t ic options

•

•

for a slightly conservative projection, take the midpoint between the smallest rate of increase (or largest rate of decline) and the multiyear average rate of change, and apply that figure to this year’s revenue. Previous rates of change, weighted toward more recent experience. Pam could identify the mean or median rate change for several years, the average rate of change for the past two years, and apply the average of these two figures to this year’s revenue.

Any of these would be reasonable, depending on how cautious or aggressive Pam wished to be with her forecast. Regardless of the approach she eventually chose, Pam planned to make adjustments to her calculations based on the current conditions she was learning about through her reconnaissance. Initially, Pam had hoped that she would discover a definitive forecasting technique—​one that would make her confident that she had “the answer.” After trying several approaches with as many different answers, however, she came to regard the various techniques—​along with Dwight’s trend line projections—​merely as inanimate advisers, each with its own recommendation. She would have to pick one and blend that advice with her own judgment. She found herself drawn to the slightly conservative approach of basing her projected rate of increase on the midpoint between the smallest rate of increase in the prior six years and the six-​year average rate of change. She would start there for both of the major—​and most stable—​revenues: property and sales taxes. Then she would adjust those amounts based on what she and her colleagues knew about current local conditions. For the other revenue categories, she would rely less on rates of change in the past than on her reconnaissance with the planning director and others—​information about what was happening right now. This strategy should yield a fairly conservative forecast for the coming year. If Pam’s projection was off the mark, she would prefer that it be lower than actual revenues, not higher.5

5 While in the throes of budget debates, other officials sometimes encourage the forecaster to provide aggressive revenue projections that will allow them to approve more of the budget requests than are possible with conservative revenue forecasts. Later in the year, however, if actual revenues fall short of aggressive projections, difficult financial decisions may become necessary and the forecaster may feel lacking in advocates and supporters. Many finance officials favor conservative forecasts.

f o re c a s t i ng

Meeting with the manager and budget officer Pam met with Michael and Jim to present her revenue forecast for the coming year. She began with the total for the general fund and then proceeded to describe and defend her projection for each major category of tax and nontax revenues.6 When Pam said that she was forecasting property tax revenues at $25,667,700, Jim busily punched numbers into his calculator. “You know, we had a 9.5 percent increase in property tax revenues from last year to this, and you’re only projecting a 1.6 percent increase for next year,” Jim said. “Isn’t that pretty low?” “Last year was the biggest jump in three years,” Pam replied. “My forecast is more in line with our history.” “Projecting another 9 or 10 percent of growth would sure make it easier for me to deal with budget requests,” Jim said. “Sorry, Jim. I just don’t think that would be wise.” “Property and sales tax projections seem pretty much in line with prior trends, but the other revenues are not. Why?” Michael asked. “Apart from property and sales taxes, most of our revenues jump around from year to year,” Pam replied. “Only those two were steady enough for me to feel confident that we were dealing with a real trend. Everything else was a little too erratic, so I felt that I should confer with planners who know what’s going on with hotel development for the occupancy tax, with the fire chief on the fees they get for filling swimming pools, and so forth.” “I see,” Michael said. “I was hoping you would be able to build a model that would forecast everything.” “Nice thought,” Pam replied, “I had hoped so, too. So far, it looks like most of my forecasting will be based on doing a lot of old-​fashioned background work on our revenue sources.”

Epilogue Pam’s projection for property taxes came within 2 percent of the actual amount received. Fortunately, the actual receipts exceeded the projection 6 Nontax revenues included fees for services.

431

432

other a nalytic options

by that percentage, so there was no need for midyear budget adjustments. Jim never mentioned his earlier attempts to get a more aggressive revenue forecast, but Pam was pretty sure that he realized her forecast had spared him a $1.6 million problem he would have had to solve. Pam’s sales tax forecast came within 3.3 percent of the actual sales tax revenues. Fortunately, this forecast, too, was on the low side. As it turned out, Misty Glen would need some excess revenues from property and sales taxes to offset a minor shortfall in other revenue categories. Pam had mixed emotions about her first stab at revenue forecasting. She was pleased that she had come close on her property and sales tax forecasts and had erred on the low side rather than the high side. She felt OK about her forecasts for other revenue categories, too, because they were close, even if she missed on the high side. “I like to think there is a science in forecasting,” Pam thought to herself, “but clearly there is some art—​and some judgment—​to it, too.”

References Blom, Barry, and Salomon A. Guajardo. Revenue Analysis and Forecasting. Chicago: Government Finance Officers Association, 2001. Frank, Howard A. Budgetary Forecasting in Local Government: New Tools and Techniques. Westport, CT: Quorum Books, 1993. Morgan, Douglas F., Kent S. Robinson, Dennis Strachota, and James A. Hough. Budgeting for Local Governments and Communities. Armonk, NY: M.E. Sharpe, 2015.

Suggested for further information Bland, Robert L., and Michael R. Overton. A Budgeting Guide for Local Government, 4th ed. Washington, DC: International City/​County Management Association, 2019. See especially “Revenue Forecasting Techniques,” p. 181. Eller, Warren S., Brian J. Gerber, and Scott E. Robinson. Public Administration Research Methods: Tools for Evaluation and Evidence-​Based Practice, 2nd ed. New York: Routledge, 2018. See “Applied Decision Tools,” Chapter 19, especially pp. 408–​413. Guajardo, Salomon A., and Rowan Miranda. An Elected Official’s Guide to Revenue Forecasting. Chicago: Government Finance Officers Association, 2000.

f o re c a s t i ng

Kong, Dongsung. “Local Government Revenue Forecasting: The California County Experience.” Journal of Public Budgeting, Accounting & Financial Management 19, no. 2 (2007): 178–​199. Nollenberger, Karl, Sanford M. Groves, and Maureen Godsey Valente. Evaluating Financial Condition: A Handbook for Local Government, 4th ed. Washington, DC: International City/​County Management Association, 2003. See 199–​202 and 209–​222. Reddick, Christopher G. “Assessing Local Government Revenue Forecasting Techniques.” International Journal of Public Administration 27, nos. 8, 9 (2004): 597–​613. Schroeder, Larry. “Forecasting Local Revenues and Expenditures.” In Local Budgeting, edited by Anwar Shah, 53–​ 77. Washington, DC: World Bank, 2007. Sun, Jinping, and Thomas D. Lynch, eds. Government Budget Forecasting: Theory and Practice. New York: Routledge, 2008. The Economist Numbers Guide: The Essentials of Business Numeracy, 6th ed. New York: Public Affairs, 2014. See “Forecasting Techniques,” pp. 96–​123. Wang, Xiaohu. Financial Management in the Public Sector: Tools, Applications and Cases. Armonk, NY: M.E. Sharpe, 2006.

Web resources For instructions and templates useful in forecasting by linear regression trend lines, see the following: www.myonlinetraininghub.com/​ e xcel-​ f unctions/​ e xcel-​ f orecast-​ l inear-​ function www.real-​statistics.com/​regression/​regression-​analysis/​ www.spreadsheetweb.com/​regression-​analysis-​better-​predictions/​ For a template and exercise associated with this chapter, see https://​tools fordecisionmaking.sog.unc.edu.

433

33 ANALYSIS OF OPERATIONS VIA BENCHMARKING

Two forms of benchmarking are found in the public sector. One form, called best practice benchmarking, has its roots in the private sector and emphasizes the adoption of practices contributing to top performance. The other form, metrics benchmarking, produces performance statistics that are compared either to performance standards or to the performance statistics of other organizations. In the corporate world, the term benchmarking refers to a well-​defined performance improvement process having a distinct lingo. This form of benchmarking, known as best practice benchmarking, is meticulously analytic and focuses not on an entire corporation, department, or program but instead on a single key process—​for instance, the acquisition of raw materials, product assembly, or product warehousing and distribution. Best practice benchmarking’s objective is to find ways to perform the selected process better. But rather than starting with a blank slate in the design of improvements, organizations engaged in best practice benchmarking seek out top performers of the chosen process—​ that is, other organizations considered to be “best in class” or “world class” in that particular activity—​ and seek their cooperation in identifying practices that account for superior

a n aly s is o f operat i o n s v i a b e n c h ma rk i ng

Decide what to benchmark

Identify partners

Gather information

Analyze

Implement for effect

Figure 33.1  The Five Stages of Best Practice Benchmarking Source: Adapted from Bengt Karlöf and Svante Östblom, Benchmarking: A Signpost to Excellence in Quality and Productivity (New York: John Wiley & Sons, 1993). Copyright © 1993 by Bengt Karlöf and Svante Östblom. Used by permission. performance. The superior results achieved by best-​in-​class performers are the benchmarks that benchmarking organizations hope to match; cooperating organizations are benchmarking partners; and the analytic endeavor is designed to identify best practices that can be adopted or, more likely, adapted for use by the benchmarking organization to improve its performance. A simple graphic depiction of the technique is shown in Figure 33.1. A somewhat more detailed description of the steps, developed for the application of best practice benchmarking in the public sector, follows: • • • • • • •

decide what to benchmark study the processes in your own organization identify benchmarking partners gather information analyze implement for effect monitor results and take further action as needed1

Best practice benchmarking has been used successfully by some local governments,2 but this type of benchmarking is much less common in the public sector than is metrics benchmarking. 1 Southern Growth Policies Board and Southern Consortium of University Public Service Organizations, “Benchmarking Best Practices,” Module 2 of Results-​Oriented Government (Research Triangle Park, NC: Southern Growth Policies Board, 1997), 5. Reprinted by permission. 2 See, for example, David N. Ammons, “Benchmarking for Performance Improvement,” in Public Productivity and Performance Handbook, 3rd ed., ed. Marc Holzer and Andrew Ballard (New York: Routledge, 2021); David N. Ammons, Ryan A. Davidson, and Ryan M. Ewalt, Development Review in Local Government: Benchmarking Best Practices (Chapel Hill: University of North Carolina, School of Government, June 2008); and Patricia Keehley, Steven Medlin, Sue MacBride, and Laura Longmire, Benchmarking for Best Practices in the Public Sector (San Francisco: Jossey-​Bass, 1997).

435

436

other a naly t ic options

Local governments engaged in metrics benchmarking—​also called results benchmarking, data benchmarking, statistical benchmarking, and comparative performance measurement—​compare their own performance statistics to performance standards or to the performance statistics of other organizations. Either of these comparisons offers a broader context for judging local performance, something that internal or year-​to-​year comparisons of performance by a single government cannot provide. Cities and counties that measure their performance in even the most rudimentary fashion and track these measures over time know whether their workload is increasing or declining from year to year. If their set of measures includes quality and efficiency measures, they also know whether their services are improving (see Chapter 3). What they do not and cannot know from year-​to-​year, single-​organization performance analysis alone is whether the quality and efficiency of their services compare favorably with the levels being achieved by other organizations. A city or county that wishes to declare that it provides high-​quality services needs more than internal comparisons to confirm that assertion. It needs reputable external pegs on which to base its claim. A local government that wishes to emphasize improvement of a given service and intends to move that service to the upper echelon of quality or efficiency needs to know what top performers elsewhere are achieving, as a benchmark to inspire its own efforts and to gauge progress. Many cities and counties that benchmark their own performance through comparison of performance statistics do so on an ad hoc basis, perhaps comparing their library’s performance statistics with national norms in one study, comparing the record of their fleet maintenance mechanics with industry standards in another, and compiling road maintenance statistics from a group of respected cities or counties for still another comparative study. Examples of ad hoc metrics benchmarking of library circulation per capita and park accessibility statistics are shown in Figures 33.2 and 33.3. Some cities and counties have joined cooperative benchmarking projects to gain similar information for a variety of local government functions and to increase the reliability of performance and cost data through uniform collection and compilation across jurisdictions.3 Local governments that assemble a set of comparative performance statistics for a given function can place their own statistics in the mix and see where they stand in the quality, efficiency, and effectiveness of services. This is

3 Cooperative projects in recent years have included the Florida Benchmarking Consortium, Michigan Local Government Benchmarking Consortium, Midwest Benchmarking Project, North Carolina Benchmarking Project, and Valley (Phoenix-​area) Benchmark Cities.

a n aly s is o f operat i o n s v i a b e n c h ma rk i ng

Figure 33.2  Benchmarking Library Circulation per Capita across Selected Cities Note: Based on FY 2017 data. Source: City and County of San Francisco, California, City Performance Scorecards: Livability Benchmarking. Accessed June 5, 2020, https://​sfgov.org/​scorecards/​benchmarking/​livability. valuable information. In some cases, this information alone may prompt service improvements. But can the analyst take these data and gain additional insights using simple analytic techniques? That is the question facing a young management analyst in the fictional city of Eliza, California.

Scenario: Eliza, California Lisa Bootmaker was looking for some bucks she could squeeze out of the budget. Better, faster, cheaper had long been the mantra among the team of

437

438

other a naly t ic options

Figure 33.3  Benchmarking Park Accessibility across Selected Cities: Percentage of Residents within One-​Half Mile of a Park Note: Based on 2017 data. Source: City and County of San Francisco, California, City Performance Scorecards: Livability Benchmarking. Accessed June 5, 2020, https://​sfgov.org/​scorecards/​benchmarking/​livability.

management analysts who occupied the cubicles in the south wing of city hall’s fourth floor, but this year better and faster were taking a back seat to cheaper. The city government was facing a budget crisis, and the analysts were desperately seeking options that could save big bucks without ravaging services. Lisa had turned her attention to the sanitation department, where the hefty budget for residential refuse collection offered a tempting target. Lisa and fellow analyst Lindy Hall liked to share stories and bounce ideas off one another. They found the practice entertaining, spiced as most of these conversations were with amusing anecdotes from the field. Lindy

a n aly s is o f operat i o n s v i a b e n c h ma rk i ng

often said that truth was stranger than fiction, but Lisa wondered how much fiction worked its way into the hilarious “truths” Lindy sometimes told. Offering more than just entertainment value, these conversations also gave the analysts a chance to air their thinking on important projects and secure feedback from a colleague who would not pull any punches when their reasoning was off base. On several occasions, each had been spared future embarrassment when an ill-​conceived conclusion had been exposed in one of these conversations and revised before it went any further. “I am leaning more and more toward recommending that we experiment with ‘managed competition’ for residential refuse services,” Lisa said, “but it is tough to guess how much we can squeeze from the budget. I’m afraid it won’t be the wad of cash I thought originally.” “By managed competition, you mean the approach where the sanitation department would compete to provide the service?” asked Lindy. “The department would submit a bid along with outside companies that are interested in the job?” “That’s right.” “Why mess with that? You want to know how to save money in refuse collection?” Lindy asked, not waiting for a response. “Just contract it out. Simple. Our sanitation department has developed a lot of bad habits over the years that have made it really inefficient. I say, just rip the Band-​Aid off and turn it over to someone else.” “I don’t think it’s quite that simple, Lindy,” Lisa replied. “It’s probably true that the sanitation department is not a model of efficiency, but shouldn’t they be given the chance to improve their operation? Several members of the city council will think they should. And if we give the work to someone else, we would have to be careful to make sure they don’t earn their profit by cutting corners and reducing the quality of service.” “Maybe you’re right, but I just know that I’ve seen plenty of reports that say contracting for refuse services is cheaper.” “I have seen those studies, too, and they say contracting tends to be cheaper. It’s no guarantee. A well-​managed in-​house operation still beats an average or weak contractor.” “Is that what we have, Lisa—​a well-​managed in-​house operation?” “I’m not sure it couldn’t be,” replied Lisa, “with proper conditions and the right incentives. That’s why I am leaning toward managed competition. Give them a chance to tighten their operation. Then, if they are the best

439

440

other a nalytic options

choice, keep the work in-​house. But if they cannot compete, turn part or all of the business over to outside companies.” “Do you have facts and figures to support this recommendation?” Lindy asked. “I think I do.”

Scatterplots and regression lines Lisa stepped back to her own cubicle and retrieved several charts. For the past several years the city of Eliza has cooperated with 11 other cities in a benchmarking project. The project is designed to produce reliable, uniformly collected, comparative performance and cost information pertaining to a variety of local government services. Lisa has used the refuse collection data from this project in her analysis. The first chart Lisa showed Lindy was a scatterplot (also called a scattergram or scatter diagram) showing the number of refuse accounts (that is, households) and the expenditures per account in Eliza and nine other cities (see Figure 33.4).4 The city of Eliza, for example, served 221,650 accounts at an expenditure rate of $92.38 per account, placing it near the center of the scatterplot. “I thought there were 12 cities in that project,” Lindy interjected. “Why are there only 10 in the scatterplot?” “These are the ones with once-​a-​week curbside service,” Lisa responded. “Including the two cities having a different service level would distort the analysis. I’m looking for expenditure differences attributable to economies of scale or operating strategies rather than to differences in service levels.” The second chart showed the scatterplot again, but this time it had a line running through it (see Figure 33.5). Lisa explained that this is a regression line drawn by the computer to depict the relationship between the two variables (number of accounts and expenditure per account) insofar as the evidence from these 10 cities is concerned. It reveals—​by the line and by an accompanying equation—​how the scale of the operation (that is, number of accounts) tends to influence expenditures per account. 4 Performance data for the city of Eliza and the other nine cities are hypothetical, but they are based on data from the North Carolina Benchmarking Project (​www.sog.unc.edu/​ resources/​microsites/​north-​carolina-​benchmarking-​project).

a n aly s is o f operat i o n s v i a b e n c h ma rk i ng

Figure 33.4  Residential Refuse Accounts and Expenditures per Account for 10 Cities

Figure 33.5  Relationship between Number of Residential Refuse Accounts and Expenditures per Account (Correlation: 0.39)

“This isn’t perfect and it isn’t especially sophisticated,” Lisa explained, “but it is informative and it’s easy to do. I prepared these charts using the regression tool in Excel. (See Appendix F for a description of the procedure for doing so.) For purposes of analysis, I consider the line to be the expected value of the expenditure per account for a given number of refuse accounts. A city can identify its expected expenditure per account simply by knowing how many accounts it is serving. Using that information, a city can

441

442

ot h e r an alyt ic options

pinpoint its spot on the horizontal line at the base of the figure. A vertical line from that point will intersect the regression line at its expected expenditure level. When a city’s actual point lies above the regression line, meaning higher than expected expenditure, it can be considered to be less efficient than average—​judging strictly from the size of the operation and ignoring all other factors. When a city’s actual point is below the regression line, it is doing something right. It is more efficient than average.”

BOX 33.1  SCATTERPLOT PATTERNS In a scatterplot, a set of data points exhibiting a positive relationship between two variables presents a pattern that moves upward from left to right—​that is, away from the horizontal base (see Figure 33.6). As one variable increases, the other variable tends to increase, too. A set of data points exhibiting neither a positive nor negative relationship between the variables is depicted by a horizontal regression line (see Figure 33.7).

Figure 33.6  Positive Relationship

Figure 33.7  No Relationship

a n aly s is of oper ati o n s v i a be n c h ma r ki n g

Figure 33.8  Negative Relationship

A set of data points exhibiting a negative relationship between the variables presents a pattern that moves downward from left to right (see Figure 33.8). As one variable increases, the other tends to decrease.

Lisa went on to explain that the analysis to this point does two things for her. First, it reassures her about citing the research literature on economies of scale in refuse collection. Previous studies indicated that refuse services enjoy economies of scale only until they reach a customer base of about 50,000 population.5 After that, costs tend to level out. All the cities in Lisa’s analysis have populations far in excess of 50,000. The smallest city serves 35,114 households (that is, accounts) and has a population of 87,789. The expenditure pattern in her cities suggests diseconomies of scale, with greater expenditures per account as the number of accounts increases. This not only strengthens her confidence in the earlier studies and in her own figures, it also supports her managed competition recommendation. Some cities using managed competition split their jurisdiction into segments of 50,000 population or greater and seek bids for each segment, knowing that this division of territory will not result in the loss of economies of scale as long as each segment has a population of at least 50,000 residents. Second, her analysis tells Lisa that the city of Eliza’s refuse service costs are well within the normal range for its customer base. In fact, Eliza’s point almost hugs the regression line for expected cost. Unlike a city with 5 E.S. Savas, The Organization and Efficiency of Solid Waste Collection (Lexington, MA: Lexington Books, 1977).

443

444

ot h e r an alyt ic options

excessive in-​house expenditures for a given service, Eliza is unlikely to reap a huge windfall in cost savings by shifting to contract service. Still, there is room for improvement, as evidenced by other cities with lower than expected expenditures. Managed competition is a strategy to secure these improvements either by streamlining in-​house operations or by contracting with a more efficient outside provider. “This is pretty cool,” Lindy remarked. “So the most efficient city is the one that is farthest below the regression line, right?” Lisa explained that for her analysis, she was using the percentage difference above or below the line rather than the raw difference. She further explained that the distance between the point and the line is called the residual and that she was looking at the residual as a percentage of the expected expenditure. So if a city is $10 below an expected expenditure of $80, which is 12.5 percent below the expected value, it is considered to be more efficient than a city with an expenditure $10 below an expected $100 per account, or 10 percent below. “But before we get too carried away with declaring this city or that one to be the most efficient, let’s remember that we are only looking at two variables here. We would undoubtedly get a more precise line with more variables using multiple regression, but I think this simple version will do the job for me.”

BOX 33.2  KEY STATISTICS RELATED TO REGRESSION The basic regression formula is: Y = a + bX where Y is the dependent variable X is the independent variable a is the Y intercept (where the regression line passes through the Y axis) b is the slope of the line Once the analyst knows the regression coefficients (the values for a and b), the value of Y can be predicted simply by plugging a given value of X into the formula.

a n aly s is of oper ati o n s v i a be n c h ma r ki n g

The correlation coefficient, r, ranges from +1.0 to −1.0 and tells the analyst the direction (positive or negative) and strength of the relationship between variables. Squaring the correlation coefficient produces the coefficient of determination, R2, which reveals the amount (percentage) of the variation in Y that can be explained by knowing X. Statistical significance tells the analyst how likely it is that the apparent relationship between variables happened simply by chance, based on a given set of observations or data points. If using traditional standards of academic research, the analyst will be looking for a significance figure of 0.05 or less (sometimes settling for 0.10 or less) before declaring the relationship to be statistically significant. Analysts using the regression technique would be wise to consult a basic statistics textbook for more complete information on the use of regression and the interpretation of regression results and statistics.

Although the scatterplot and regression line were helpful components of Lisa’s analysis, stopping at this point would have fallen short of what Lisa really wanted to know. “If I am going to recommend managed competition—​or contracting, for that matter—​I would like to have some reassurance that there is a high probability of a favorable result.” Lisa presented another chart (see Figure 33.9). Each residual, expressed as a positive or negative percentage from the expected expenditure, was shown in a bar graph. The bars depicting unfavorable results (that is, positive residuals) appear above the horizontal line. The bars depicting favorable results (that is, greater efficiency reflected by negative residuals) appear below the horizontal line. Among the data collected in the benchmarking project was information on the extent of contracting for refuse services—​specifically, the percentage of all refuse accounts served by contract. Each of the bars in Figure 33.9 is coded to reflect the extent of residential refuse collection contracting in that city. Only Eliza and two others among the 10 cities handle all or almost all collection using city employees. Two cities handle everything by contract. The other five cities operate with split systems—​part in-​house and part by contract. “I was hoping this chart would give me a clearer answer regarding ‘best practice,’ ” Lisa said. “Still, it is helpful. Most of all, it cautions me not to

445

446

ot h e r an alyt ic options

Figure 33.9  Comparing Actual to Expected Expenditures for Refuse Collection among Cities with Various Levels of Contract Services

expect contracting to be a magic potion. Sure, the two cities that contract their refuse collection entirely have favorable residuals (lower than expected expenditures), but another city that contracts almost everything has much higher than expected expenditures. And two cities that operate mostly in-​ house have great results!”

BOX 33.3  OTHER APPLICATIONS OF BENCHMARKING Any local government operation could benefit from careful benchmarking. An animal control unit, for example, might discover that its pet adoption rates are far lower than those of some of its counterparts, thus leading to a series of conversations that reveal the other unit’s secrets to success. A sanitation department might discover that the tonnage of refuse collected by its average worker is much less than the comparable figure for its benchmarking partners, with further analysis revealing that its equipment is outmoded or its collection routes poorly designed.

a n aly s is of oper at i o ns v i a be n c h ma r ki n g

The city and county of San Francisco routinely posts on its website Performance Scorecards for community livability, the environment, public health, public safety, transportation, safety net programs, government finance, and the local economy (https://​sfgov.org/​scorecards/​). In addition to displaying historical trends in San Francisco’s figures, the scorecard displays benchmarking data compiled annually by its Controller’s Office. For example, benchmarking comparisons with 16 peers showed that the San Francisco Municipal Transit Agency (SFMTA) was among the strongest performers in maintaining a low expense per passenger trip (Figure 33.10). If SFMTA was hoping to find ideas among its peers for ways to reduce maintenance and operating expenses, the benchmarking data pointed only to San Diego, Long Beach, Chicago, and Boston as likely prospects for such ideas. With a farebox recovery ratio of 25 percent, the SFMTA was near the middle of the group in the portion of transit expenses recovered through fares. The public safety scorecard reports how San Francisco compares to peer jurisdictions across performance metrics for violent crime, property crime, and motor vehicle break-​ins. A scatterplot posted on the website shows that San Francisco had a lower-​than-​average rate of violent crime but a greater-​than-​average rate of property crime (Figure 33.11).

Postscript Lisa prepared a report recommending that the city proceed toward managed competition for refuse services. She incorporated her charts into the report and noticed with some pleasure the city manager’s surprise at the absence of economies of scale among large service providers and the surprise of two business executives on the city council at the absence of clearer evidence in support of contracting for services. Four others joined these two in a 6–​3 vote to move gradually toward managed competition, beginning with a single quadrant of the city. One council member, who previously had been considered pro-​privatization, said he was now convinced that competition, rather than privatization, is the key to good performance and that he hoped the sanitation department could develop the winning bid. Funds were

447

newgenrtpdf

448 other a naly t ic options

Figure 33.10  Benchmarking Public Transit Operations across Selected Cities Note: Based on 2017 data. Source: City and County of San Francisco, California, City Performance Scorecards: Livability Benchmarking. Accessed June 5, 2020, https://​sfgov.org/​scorecards/​transportation-​benchmarking.

newgenrtpdf

a n aly s is o f operat i o n s v i a b e n c h ma rk i ng

Figure 33.11  Scatterplot of Violent and Property Crime Rates in Selected Cities

449

450

other a nalytic options

appropriated to assist the sanitation department in reviewing its operation, evaluating equipment and deployment practices, and preparing its bid. Projected savings from managed competition in refuse collection were pegged at a modest level. The search for major budget savings continued elsewhere. Lisa’s management analyst colleague Lindy was persuaded by the charts and statistics that sorting out the promises and benefits of contracting is a lot more complicated than he thought originally. He backed away from his blind advocacy of that option.

References Ammons, David N. “Benchmarking for Performance Improvement.” In Public Productivity and Performance Handbook, 3rd ed., edited by Marc Holzer and Andrew Ballard. New York: Routledge, 2021. Ammons, David N., Ryan A. Davidson, and Ryan M. Ewalt. Development Review in Local Government: Benchmarking Best Practices. Chapel Hill: University of North Carolina, School of Government, June 2008. Karlöf, Bengt and Svante Östblom. Benchmarking: A Signpost to Excellence in Quality and Productivity. New York: John Wiley & Sons. 1993. Keehley, Patricia, Steven Medlin, Sue MacBride, and Laura Longmire. Benchmarking for Best Practices in the Public Sector. San Francisco: Jossey-​ Bass, 1997. Savas, E.S. The Organization and Efficiency of Solid Waste Collection. Lexington, MA: Lexington Books, 1977. Southern Growth Policies Board and Southern Consortium of University Public Service Organizations. “Benchmarking Best Practices,” Module 2 of Results-​Oriented Government. Research Triangle Park, NC: Southern Growth Policies Board, 1997.

Suggested for further information Ammons, David N. Municipal Benchmarks: Assessing Local Performance and Establishing Community Standards, 3rd ed. Armonk, NY: Routledge/​M.E. Sharpe, 2012. Camp, Robert C. Benchmarking: The Search for Industry Best Practices that Lead to Superior Performance. Milwaukee, WI: Quality Press, 1989.

a n aly s is o f operat i o n s v i a b e n c h ma rk i ng

Eller, Warren S., Brian J. Gerber, and Scott E. Robinson. Public Administration Research Methods: Tools for Evaluation and Evidence-​Based Practice, 2nd ed. New York: Routledge, 2018. See “Simple Linear Regression,” Chapter 17, 351–​374, and “Applied Decision Tools,” Chapter 19, especially 414–​418. Gibilisco, Stan. Statistics Demystified. New York: McGraw-​Hill, 2004. Gupta, Dipak K. Decisions by the Numbers: An Introduction to Quantitative Techniques for Public Policy Analysis and Management. Englewood Cliffs, NJ: Prentice-​Hall, 1994. Meier, Kenneth J., Jeffrey L. Brudney, and John Bohte. Applied Statistics for Public and Nonprofit Administration, 9th ed. Stamford, CT: Cengage Learning, 2015. Spendolini, Michael J. The Benchmarking Book, 2nd ed. New York: AMACOM, 2000. Taylor, Bernard W., III. Introduction to Management Science, 9th ed. Upper Saddle River, NJ: Prentice-​Hall, 2006.

Web resources Florida Benchmarking Consortium www.flbenchmark.org/​ International City/​County Management Association, “Open Access Benchmarking” https://​icma.org/​open-​access-​benchmarking Local Government Benchmarking Framework ​www.improvementservice.org. uk/​benchmarking North Carolina Benchmarking Project ​www.sog.unc.edu/​resources/​microsites/​ north-​carolina-​benchmarking-​project Also, consider a web search using these key words: local government benchmarking, performance benchmarking, and best-​ practice benchmarking.

451

Part VII PRESENTING ANALYTIC EVIDENCE

When all the data have been collected and all the analysis done, how should the evidence be presented? Tables and graphs are popular choices for presenting analytic evidence, but which of these is the better choice? And when is one type of graph more suitable than another? Chapter 34 offers some answers.

34 PRESENTING ANALYTIC EVIDENCE TABLES OR GRAPHS?

When local government managers and analysts present fact-​ based recommendations, they often include tables of numbers to support their findings and back up their proposed course of action. The tables might, for example, show revenues and expenditures over a span of several years, the performance measures for a particular function over a period of time or compared to other cities or counties, or relevant numbers from the review of some other topic of concern. Although the tables often are accompanied by text that explains or draws attention to key numbers, sometimes the tables stand alone. In these instances the manager or analyst presumes that the numbers speak for themselves. In reality, there are times when a table of numbers is the best choice for presenting analytic evidence and times when it is not. Even when a table is the best choice, the table’s design can make the difference between an effective presentation and an ineffective one. Sometimes a graph is a better choice—​ but not just any graph. A pie chart or bar graph can work well in some cases but be a poor choice in others.

456

presentin g analy tic ev i d ence

Scenario: Dulles, Texas Dawn Meredith, director of research and budget, is considered by most observers to be a rising star in the municipal government of Dulles. She was handpicked for the job by longtime city manager Tom Lundee from a national pool of good candidates and, after just two years on the job, Dawn has established a reputation at city hall for insightful assessments and good judgment. At the time of Meredith’s appointment, Lundee had changed the department’s name from the budget office to the research and budget department. He hoped that the office’s focus could be expanded from its traditional role of reviewing budget requests and monitoring budget transactions to now include a new emphasis on data analysis and assessment. Lundee emphasized the importance of upgrading and expanding the department’s research products. “I need you to quarterback that effort,” he had said. Dawn embraced her role. Soon her department was taking on a variety of analytic projects and introducing a greater level of data-​driven decision making into the budget process. Now two years into her tenure, the research and budget team was gathering for one of its biweekly staff meetings. “Good morning, everybody,” Dawn began. “Let’s get started.” The meeting moved quickly through updates on a host of budget issues facing the city and status reports on current analytic projects. Before wrapping up, Dawn gave the team a new assignment. “We should all be pleased with the progress we’ve made in introducing more analysis into the city’s decision-​making processes,” she said. “But in some cases, I think we may be shortchanging our analyses by ineffective presentation of our findings.” She paused to see if there was any response. Melinda Renfro, often the most outspoken of the team’s four budget analysts, reacted defensively. “What’s wrong with the way we present our analysis? We’ve introduced more tables and graphs in our reports and in the budget document than ever before! That seems pretty good to me.” Dawn waited for other responses. After several quiet seconds, Kelvin Hill broke the silence. “Maybe it’s not a matter of whether we have tables and graphs in our work. Maybe it’s whether we’re making the best choices among tables and graphs. We should be asking ourselves whether the tables and graphs we are using depict our findings well.”

tabl e s o r gr a p h s ?

“Exactly!” Dawn replied. “I’ve noticed several occasions when people seem not to have grasped findings that we presented to them only a week or two earlier. They make comments totally at odds with our findings or raise questions that our findings answered just a week earlier. Somehow our analysis didn’t sink in. I’m wondering if we are not presenting our work as effectively as we could. I have revisited last year’s budget document and a few of our recent reports, and I have some thoughts on possible improvement. But first I want your ideas.” Dawn asked the four budget analysts and the department’s intern to think about the presentation of analytic evidence and come together as a team to make some recommendations to her. She gave the team three weeks to do this.

The analysts respond As the meeting broke up, Kelvin called out to his fellow analysts to halt their exodus from the room. “Huddle up, everybody! This will just take a minute. We only have three weeks, so let’s meet again the day after tomorrow.” When everyone nodded in agreement, Kelvin continued, “Between now and then, take a look at several of our recent reports and last year’s budget document. Do a quick critique of our presentation of analytic evidence and come to the meeting two days from now prepared to offer recommendations for how we do it in the future. Does that sound okay?” A couple of analysts mumbled something about other pressing assignments, but no one otherwise objected to the plan. When the group reassembled two days later, Melinda Renfro was still defensive about the idea that changes needed to be made. “I think we do a good job of presenting our evidence. We have lots of tables and great pie charts! Maybe we could use a few more pie charts. And maybe more of them should be in color.” Melinda had become a great fan of pie charts as soon as she learned a few years back how easily they could be produced. They quickly became her go-​to graphics. “Everything doesn’t have to be presented as a pie chart,” said analyst Lilly Roberts, stopping Melinda in her tracks. “There are other options, you know.”

457

458

presentin g analy tic ev i d ence

The group brainstormed presentation ideas and discussed a wide range of options, based primarily on graphics the analysts had seen in reports and presentations by other cities and counties. After about an hour, they agreed to get back together in a week after individually developing a few sample tables or graphics. Before breaking up, analyst Lee Roy Gordon added, “While the rest of us are developing tables and graphics based on what we’ve seen elsewhere, why don’t we get Roger to see if there’s any literature that provides some advice about this?” The Roger he referred to was Roger Starbuck, the department’s intern. “Good idea,” Kelvin and others replied. The analysts met the next week and once more after that before finalizing their recommendations to Dawn Meredith, the research and budget director. Melinda recommended more pie charts and perhaps brighter colors whenever color was an option. She had samples using Dulles data to show what she had in mind. Kelvin thought they might be overdoing pie charts and came with examples using line graphs and bar charts. “C’mon, Melinda,” he said, “not everything lends itself to a pie chart.” Lilly thought that tables should be expanded to show more years of history. “That will allow readers to pick out trends in the numbers,” she said. Lee Roy promoted greater use of histograms or bar charts and brought a few samples using Dulles data. “I used a 3-​D feature on one of these,” he said, “and it practically jumps off the page! That’ll get the reader’s attention.” When the analysts asked Roger whether he had found any good advice in the literature, he replied, “Actually, I learned that there are a lot of presentation options. You guys are focusing on some of the most popular choices, but there are others, too. The basic advice seems to be that the choice of presentation mode should be matched to its purpose.” “OK,” replied Kelvin, “well, our purpose right now is to pick some of these tables and graphs and get ready for our meeting with Dawn.” The analysts gave little heed to the intern’s advice and proceeded to agree on several of the most promising tables and graphs for presentation to their boss.

tabl e s o r gr a p h s ?

Guidelines for the presentation of analytic evidence Roger Starbuck’s reconnaissance warranted greater attention from the analysts than they gave it. He unearthed some good advice regarding the use of tables and graphs to present analytic evidence. The analysts should have probed for more details. But then, the department’s veterans do not always listen to the intern. Some of the best advice Roger found on the presentation of analytic evidence is provided in Table 34.1 and Figure 34.1. Choosing a table rather than a graph or a graph rather than a table is not simply a matter of style or taste. Tables are better suited to some purposes and graphs to others. A table presents the reader with rows and columns of precise information. Someone seeking a precise value for comparison or further analysis is more likely to find it in a table than in a graph. For that reason, reference documents typically rely heavily on tables as their principal mode of presentation. Graphs are a better choice when the purpose is to illustrate a trend or the relationships among variables. They also can help the reader visualize differences in a comparison. Graphs are the better choice when “the message is contained in the shape of the values.”1 Once the choice is made between a table and a graph, still more decisions await. Choices in the formatting of a table—​for instance, the use of white space and shading—​can make the table easier to read. The decision to use a graph introduces an array of options. Choices among the types of graphs can make the difference between an effective presentation of analytic evidence and an ineffective one. The analysts paid no attention to any of this.

The analysts present their proposal When the analysts met with research and budget director Dawn Meredith, they projected a series of slides on the screen in the conference room to demonstrate how they would improve the presentation of the department’s

1 Stephen Few, Show Me the Numbers: Designing Tables and Graphs to Enlighten (Oakland, CA: Analytics Press, 2004), 239.

459

460

presentin g analy tic ev i d ence

Table 34.1  General Tips for Tables and Graphs Not all messages about data warrant a table or figure. In some cases—​especially when the analytic results are simple—​a sentence or two, perhaps with data summarized parenthetically, may be a better choice. Use this rule of thumb: If the message can be clearly expressed in a couple of sentences, use text. A table or graph in such cases is probably unnecessary. Tables present columns of numbers or text, with each column having a title or label. Large tables with multiple rows and columns of numbers can be difficult to read. Assist the reader by the judicious use of white space (for instance, after every fifth row of numbers) or by using subtle fill color to lightly shade alternating rows or alternating columns or to draw attention to the particular table cells of interest. If a table extends beyond a single page, repeat column headings on each subsequent page. In choosing between tables and graphs… • Pick tables when the reader will need to see precise values or when the purpose is to communicate more than one unit of measure. • Pick graphs when the objective is to reveal relationships among variables, to illustrate trends, to make comparisons, or when the findings from the data can best be shown by the shape of a graph created from the data. When designing graphs strive for accuracy, clarity, and simplicity. Choose images the viewer will grasp easily and interpret clearly. In graphs use line width, color, dashed lines, boldface or other techniques to draw the reader’s attention to points of emphasis. Bar charts that have something other than zero as their starting point can be misleading. Differences between the bars can seem greater than they actually are. It is usually best to start at zero to avoid this problem; but if space considerations preclude the use of a zero starting point, be sure to include a note alerting the reader to the adjusted starting point. Having a non-​zero starting point is not a problem for line charts, as the reader’s eye is not drawn to the dimension of size as with bar charts. When two variables are strongly correlated, a scatterplot of all observations can be used to illustrate that relationship—​as one variable increases, the other increases (positive correlation) or decreases (negative correlation). Sometimes a regression line or trend line can be added to the scatterplot to help the reader see the relationship more clearly. Avoid the use of 3-​D graphics. The illusion of a third dimension often causes a distortion of the data that makes it difficult for the reader to discern the values of the depicted variables. Tables and figures should be sufficiently clear, well-​labeled, and described by their caption or legend to stand alone and be interpretable, whether the audience reads the accompanying narrative in the report or not. Sources: Adapted from multiple works, especially “Almost Everything You Wanted to Know about Making Tables and Figures,” Bates College Department of Biology, 2012. Accessed October 25, 2020http://​abacus.bates.edu/​~ganderso/​biology/​resources/​writing/​HTWtablefigs.html; Stephen Few, Show Me the Numbers: Designing Tables and Graphs to Enlighten (Oakland, CA: Analytics Press, 2004); Marisa Krystian, “Do You Know When to Use Tables vs. Charts?” Infogram.com, 2016. Accessed October 25, 2020 https://​infogram.com/​blog/​do-​you-​know-​when-​to-​use-​tables-​vs-​charts/​; Disha Misal, “Charts vs Tables: How To Best Represent Your Data,” AnalyticsIndiaMag.com, 2019. Accessed October 25, 2020.https://​analyticsindiamag.com/​charts-​vs-​tables-​how-​to-​best-​represent-​your-​data/​; and University of North Carolina at Chapel Hill, Writing Center, “Figures and Charts.” Accessed October 25, 2020. https://​writingcenter.unc.edu/​tips-​and-​tools/​figures-​and-​charts/​.

tabl e s o r gr a p h s ?

If depicting the relationship between two variables… Consider a scatterplot

…without a line

…with a line

If depicting shares of the total… Consider a pie chart for a single point in time

Consider a stacked column chart for changes over time

If depicting the distribution of a single variable …having few data points

…having many data points

Consider a bar chart (histogram)

Consider a line histogram

If comparing one variable per item and …many items

…only a few items

Consider a horizontal bar chart

Consider a column chart

Figure 34.1  Choosing the Right Chart or Graph

461

462

presentin g analy tic ev i d ence

If depicting a variable over time …for a single category

…for multiple categories

Consider a line chart

Consider a line chart

If comparing the central tendency and distribution of a variable across time or across different units Consider a box plot

If showing how data items vary across geography or space Consider a shaded map, with darkness indicating the variable’s intensity

Figure 34.1 Continued

tabl e s o r gr a p h s ?

If showing how performance compares to a standard or benchmark Consider a chart combining bars and lines

Figure 34.1 Continued Sources: Based on multiple works, especially Stephen Few, Show Me the Numbers: Designing Tables and Graphs to Enlighten (Oakland, CA: Analytics Press, 2004); www.PerceptualEdge. com; Stephanie D.H. Evergreen, Effective Data Visualization:The Right Chart for the Right Data (Thousand Oaks, CA: Sage Publications, 2017); Zach Gemignani, Chris Gemignani, Richard Galentino, and Patrick Schuermann, Data Fluency: Empowering Your Organization with Effective Data Communication (Indianapolis, IN: John Wiley and Sons, 2014); and www.ChartChooser.com.

analytic evidence. Of the dozen slides, seven were tables and five were graphs. Dawn offered comments as they moved through the slides. With each slide she asked, “What is the message here?” In some cases, the analysts had a quick and confident response; but in others, they were silent or offered only a halting reply. When they came to one of Melinda’s pie charts and Dawn asked her question about the chart’s message, Melinda said, “This shows how much of the work at fleet maintenance is directed toward preventive maintenance and how much goes to repair work in the shop and to emergency repair work following breakdowns out on the road (Figure 34.2).” “So the message is, what?” “As we put less into preventive maintenance,” replied Melinda, “we can expect to see more repair work, including more repairs out on the road.” “Pie charts give a snapshot of one point in time,” said Dawn. “Don’t you need to see a trend rather than a point in time in order for the reader to receive the message you want them to receive? Can you actually hope to depict a causal relationship with just a pie chart?”

463

464

presentin g analy tic ev i d ence

“I suppose you’re right. Maybe we should have two or three pie charts showing different points in time.” Melinda really likes pie charts. “Or perhaps you should consider a different type of graph. Take a look at using a vertical bar chart having a cluster of three bars depicting your three categories two years ago, last year, and this year,” Dawn suggested. “Or look at a stacked column chart showing those three years.” Following these suggestions, the analysts produced the graphs shown in Figures 34.3 and 34.4. Fleet Department Work Orders by Type Preventive Maintenance 29%

Repair Work 59% Emergency Repair 12%

Figure 34.2  What the Analysts Initially Prepared to Show Fleet Department Work

Figure 34.3  What the Analysts Prepared to Better Show the Trends in Fleet Work: Column Chart

tabl e s o r gr a p h s ?

Figure 34.4  What the Analysts Prepared to Better Show the Trends in Fleet Work: Stacked Column Chart

She turned to the other members of the analyst team. “You also used a pie chart to show the various sources of our revenue—​property tax, sales tax, fees for services, and intergovernmental revenue—​and the share of total revenue each represents (Figure 34.5),” Dawn said before asking her now-​standard question. “What’s the message?” “We are very dependent on property tax revenues,” Melinda replied. “The pie chart shows that.” “True, but it doesn’t show how much more dependent we’ve become on sales tax revenues in recent years,” Dawn said. “Or on fees for services,” Lilly interjected. “Consider replacing this with a line graph that extends four or five years and has a line for each of the revenue sources,” Dawn suggested. “See if that conveys a more complete message.” “I told you we should have dropped the pie charts,” Lee Roy whispered to Kelvin. “One more point I would like to make,” added Dawn. “We might still be a little table-​heavy. See if there’s a message we want to convey on a couple of the tables that could be better depicted by a graph. And for the remaining tables, consider whether things could be done to make them easier to read (Table 34.2).”

465

466

presentin g analy tic ev i d ence

Municipal Revenues for 2019 Sales Tax, $28,74,741

Intergovernmental, $6,27,731

Fees, $9,17,919

Property Tax, $82,27,622

Other, $10,39,724

Property Tax

Sales Tax

Intergovernmental

Fees

Other

Figure 34.5  City of Dulles Revenue by Source, 2019

Table 34.2  Initial, Tightly Formatted Table Prepared by Analysts to Show Municipal Staff by Department Municipal positions by department 2013 2014 2015 2016 2017 2018 Police Fire Public Works HR

2019

40 34 28 2

42 34 25 2

45 34 27 2

48 36 29 2

50 37 30 2

50 38 30 3

52 40 32 3

Finance

6

6

7

7

8

8

8

Administration Planning

5 4

5 5

6 5

6 6

6 6

6 6

6 8

Parks

8

10

12

14

15

15

15

Fleet

3

4

4

4

4

5

5

130 133 142 152 158

161

169

Total

tabl e s o r gr a p h s ?

Revised plan for presenting analytic evidence One week later the analysts presented an improved set of slides. The charts conveyed their messages clearly and powerfully. The tables were easier to read (Table 34.3). Table 34.3  What the Analysts Prepared to Make the Table More Readable Municipal positions by department 2013

2014

2015

2016

2017

2018

2019

Police

40

42

45

48

50

50

52

Fire

34

34

34

36

37

38

40

Public Works

28

25

27

29

30

30

32

HR

2

2

2

2

2

3

3

Finance

6

6

7

7

8

8

8

Administration

5

5

6

6

6

6

6

Planning

4

5

5

6

6

6

8

Parks

8

10

12

14

15

15

15

Fleet

3

4

4

4

4

5

5

Total

130

133

142

152

158

161

169

Municipal Revenue by Source and Year Property Tax

Sales Tax

Intergovernmental

Fees

Other

$9M $8M $7M $6M $5M $4M $3M $2M $1M $0M

2017

2018

2019

Figure 34.6  What the Analysts Prepared to Better Show the Trends in Revenue: Line Chart

467

468

presentin g analy tic ev i d ence

Municipal Spending per Capita: Current Dollars and Inflation-Adjusted Dollars Spending per capita in current dollars Inflation-adjusted (constant 2012 dollars) $580 $560 $540 $520 $500 $480 $460

2012

2013

2014

2015

2016

2017

2018

2019

Figure 34.7  What the Analysts Prepared to Show Efficiency in Spending over Time

“I like this a lot,” Dawn exclaimed, “and I predict that Tom Lundee will like it, too. In fact, I am looking forward to presenting these slides to him as examples of the direction we are moving. Anyone want to join me in that meeting?” Five hands shot up in response. Clearly, everyone was pleased with the improvements. “Just one more thing before I schedule that meeting,” Dawn added. “We’ve all heard the comments at budget time about how city expenditures keep rising from year to year, which is true. But I would like to see a line graph that shows some history of expenditures per capita—​not just total expenditures—​and let’s present them in constant dollars (Figure 34.7). Let’s see what message that graph will convey about our efficiency.”

References Bates College Department of Biology. “Almost Everything You Wanted to Know about Making Tables and Figures,” Bates College Department of Biology, 2012. Accessed October 25, 2020.http://​abacus.bates.edu/​~ganderso/​ biology/​resources/​writing/​HTWtablefigs.html. Evergreen, Stephanie. Effective Data Visualization: The Right Chart for the Right Data. Thousand Oaks, CA: Sage Publications, 2017.

tabl e s o r gr a p h s ?

Few, Stephen. Show Me the Numbers: Designing Tables and Graphs to Enlighten. Oakland, CA: Analytics Press, 2004. Gemignani, Zach, Chris Gemignani, Richard Galentino, and Patrick Schuermann. Data Fluency: Empowering Your Organization with Effective Data Communication. Indianapolis, IN: Wiley, 2014. Krystian, Marisa. “Do You Know When to Use Tables vs. Charts?” Infogram. com, 2016. Accessed October 25, 2020. https://​infogram.com/​blog/​do-​ you-​know-​when-​to-​use-​tables-​vs-​charts/​. Misal, Disha. “Charts vs Tables: How to Best Represent Your Data,” AnalyticsIndiaMag.com, 2019. Accessed October 25, 2020.https://​ analyticsindiamag.com/​charts-​ vs-​tables-​ how-​ to-​ best-​ represent-​ your-​ data/​. University of North Carolina at Chapel Hill, Writing Center. “Figures and Charts.” Accessed October 25, 2020. https://​writingcenter.unc.edu/​tips-​ and-​tools/​figures-​and-​charts/​.

Suggested for further information Eller, Warren S., Brian J. Gerber, and Scott E. Robinson. Public Administration Research Methods: Tools for Evaluation and Evidence-​Based Practice, 2nd ed. New York: Routledge, 2018. See “Presenting Research: Writing and Speaking in Policy and Management Settings,” Chapter 22, 463–​481. Grob, George F. “Writing for Impact.” In Handbook of Practical Program Evaluation, 4th ed., edited by Kathryn E. Newcomer, Harry P. Hatry, and Joseph S. Wholey, 739–​764. Hoboken, NJ: Jossey-​Bass, 2015. Knaflic, Cole Nussbaumer. Storytelling with Data: A Data Visualization Guide for Business Professionals. Hoboken, NJ: Wiley, 2015.

Web resources JuiceAnalytics, “Create Interactive Web Apps to Visualize Your Data.” www. juiceanalytics.com/​ Perceptual Edge, “Visual Business Intelligence for Enlightening Analysis and Communication.” www.perceptualedge.com/​ Evergreen Data, “Intentional Reporting and Data Visualization.” https://​ stephanieevergreen.com/​

469

Part VIII WRAP UP

Carefully conducted analyses can contribute to well-​informed decisions in local government. Such analyses often can help a government seize an opportunity or avoid a serious misstep. Nevertheless, other factors sometimes carry the day, leading decision makers to deviate from analytically based recommendations. Analysts who are committed to their craft and determined to bring analysis to bear on local government decisions do not simply give up in the wake of occasional disappointments, frustrated that their recommendations were not followed. Instead, dedicated analysts enjoy their successes, accept their disappointments, learn from both kinds of experience, and look forward to the next opportunity to apply their analytic skills.

35 THE PLACE OF ANALYSIS IN A POLITICAL ENVIRONMENT

If analysts or analytically oriented managers expect facts, figures, and logic always to prevail in the decision-​making process, they have many surprises awaiting. Local governments are not businesses; their decisions are not made under the influence of a single “bottom-​line” objective. Local government decision makers must contend with multiple and often conflicting objectives. Efficiency, for example, normally is presumed to be one of the objectives, but rarely the only objective. Political objectives, as well as concerns for responsiveness, equity, and effectiveness, may override the influence of efficiency. Careful analysis geared toward identifying the most efficient method of performing a particular function or yielding the greatest return on investment may produce recommendations short on political sensitivity, with little chance of approval. For example, the most efficient methods of dealing with delinquent taxpayers, speeding motorists, stray dogs, uncooperative developers, and belligerent service recipients might veer from the community’s values and appropriately be rejected by government officials. Similarly, the most logical decisions regarding the fate of a dilapidated building, the signalization of a

474

w rap up

street intersection, or the siting of an airport or landfill may not be politically acceptable.

Disappointment, yes; discouragement, no Analysts who have devoted hundreds of hours to a project only to see their recommendations rejected are sure to feel disappointment. Allowing their disappointment to be transformed into discouragement or cynicism, however, could deprive the decision-​making process of an important element, if the discouraged analyst swears off similar efforts in the future. Astute analysts and savvy managers understand that any of a variety of factors affecting the decision-​making process can influence the eventual decision. Personal views or biases, loyalties, political allegiances, or emotion can be as potent as analytical insights in swaying individual or collective choices. In the face of disappointment, analysts must look critically at their own work and be honest about the strength of its conclusions and the influence it deserved to wield. Sometimes, analytically based arguments do not deserve to win! Sometimes, analysis is poorly performed or otherwise produces faulty recommendations—​for quantitative analysis is not infallible. Even the product of carefully conducted, thorough analysis will not convince every audience or carry every decision, but sometimes it can be influential. Sometimes it will prevail. A discouraged manager who on the heels of a rejection of sound analytical advice vows never again to expend the personal and staff resources to conduct elaborate analysis would, if faithful to that vow, deprive the decision-​making process of ever again having a chance for that influence. Thoughtful analysis normally elevates the quality of debate on an issue. Important aspects of a decision and its likely ramifications become more apparent. Sometimes the analysis is so compelling that its recommendations are adopted in full; sometimes recommendations are modified to be more politically acceptable, but are nevertheless adopted with most characteristics intact. Even when its recommendations are rejected, careful analysis permits decision makers to proceed with eyes wide open, fully aware of the likely consequences of a particular decision.

a n aly s is in a p o l i t i cal e n v i ro n m e n t

Occasional rejections of recommendations from analytic efforts should be neither surprising nor discouraging. That other elements influence local government decisions is simply a fact of life. The wise analyst, however, will not simply dismiss rejections as “their problem,” scoffing at the shortsightedness of decision makers. Analysts who hope to learn from their experiences—​bad and good—​will challenge their own analyses: • • • • •

• • •

Did I use appropriate methods of analysis? Were the assumptions underlying the analysis reasonable? Were my recommendations logical, given my findings? Did I explain my analysis—​ including methods, findings, and recommendations—​clearly? Did I anticipate the factors that were most influential to the ultimate decision? Did I offer alternative recommendations that accommodated most of those factors? Were my recommendations reasonable and feasible? Was my work completed in a timely fashion? Whether my recommendations were approved or not, how could the analysis, interpretation, recommendations, report, and presentation have been improved?

Analysts committed to improving their product with each new opportunity can expect to achieve a respectable batting average for approvals. Still, they will strike out on occasion, and it will do no good to throw their bat when they do. It may ease the frustration of an analyst still smarting from a recent rejection to recall that no one with several turns at the plate has ever completed a career still hitting 1.000.

475

Appendix A DISTRIBUTION OF CHI-​SQUARE

The chi-​square table shows, for a given combination of degrees of freedom (df) and probability (p), the minimum point of chi-​square significance. For example, with 20 degrees of freedom and a probability of .05, the chi-​square statistic would need to be at least 31.410 to be significant. That is the point at which the df row of 20 and the probability column of 0.05 intersect.

dis t ri but i o n o f c h i -s qua re

p=

0.50

0.30

0.20

0.10

0.05

0.02

0.01

0.001

df 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

0.455 1.386 2.366 3.357 4.351 5.348 6.346 7.344 8.343 9.342 10.341 11.340 12.340 13.339 14.339 15.338 16.338 17.338 18.338 19.337 20.337 21.337 22.337 23.337 24.337 25.336 26.336 27.336 28.336 29.336

1.074 2.408 3.665 4.878 6.064 7.231 8.383 9.524 10.656 11.781 12.899 14.011 15.119 16.222 17.322 18.418 19.511 20.601 21.689 22.775 23.858 24.939 26.018 27.096 28.172 29.246 30.319 31.391 32.461 33.530

1.642 3.219 4.642 5.989 7.289 8.558 9.803 11.030 12.242 13.442 14.631 15.812 16.985 18.151 19.311 20.465 21.615 22.760 23.900 25.038 26.171 27.301 28.429 29.553 30.675 31.795 32.912 34.027 35.139 36.250

2.706 4.605 6.251 7.779 9.236 10.645 12.017 13.362 14.684 15.987 17.275 18.549 19.812 21.064 22.307 23.542 24.769 25.989 27.204 28.412 29.615 30.813 32.007 33.196 34.382 35.563 36.741 37.916 39.087 40.256

3.841 5.991 7.815 9.488 11.070 12.592 14.067 15.507 16.919 18.307 19.675 21.026 22.362 23.685 24.996 26.296 27.587 28.869 30.144 31.410 32.671 33.924 35.172 36.415 37.652 38.885 40.113 41.337 42.557 43.773

5.412 7.824 9.837 11.668 13.388 15.033 16.622 18.168 19.679 21.161 22.618 24.054 25.472 26.873 28.259 29.633 30.995 32.346 33.687 35.020 36.343 37.659 38.968 40.270 41.566 42.856 44.140 45.419 46.693 47.962

6.635 9.210 11.345 13.277 15.086 16.812 18.475 20.090 21.666 23.209 24.725 26.217 27.688 29.141 30.578 32.000 33.409 34.805 36.191 37.566 38.932 40.289 41.638 42.980 44.314 45.642 46.963 48.278 49.588 50.892

10.828 13.816 16.266 18.467 20.515 22.458 24.322 26.124 27.877 29.588 31.264 32.909 34.528 36.123 37.697 39.252 40.790 42.312 43.820 45.315 46.797 48.268 49.728 51.179 52.620 54.052 55.476 56.892 58.301 59.703

Single right-​tailed probability of the chi-​squared distribution

477

Appendix B DISTRIBUTION OF t

The t-​distribution table shows the threshold value at which a t-​statistic reaches statistical significance for a given combination of degrees of freedom (df) and probability (p). For example, with 20 degrees of freedom and a probability of .05 for a one-​tailed test (or .10 for a two-​tailed test), the threshold value for the t-​statistic is 1.72. This value is found where the df row and p column intersect. For a one-​tailed test a t-​value of 1.72 or greater would be significant at p