Multicriteria Decision Making: Systems Modeling, Risk Assessment and Financial Analysis for Technical Projects 3110765640, 9783110765649

Citation preview

Timothy Havranek, Doug MacNair Multicriteria Decision Making

Also of interest Value-Based Engineering. A Guide to Building Ethical Technology for Humanity Sarah Spiekermann,  ISBN ----, e-ISBN (PDF) ----

Technology Development. Lessons from Industrial Chemistry and Process Science Ron Stites,  ISBN ----, e-ISBN (PDF) ----

Engineering Innovation. From idea to market through concepts and case studies Benjamin M. Legum, Amber R. Stiles, Jennifer L. Vondran,  ISBN ----, e-ISBN (PDF) ----

Empathic Entrepreneurial Engineering. The Missing Ingredient David Fernandez Rivas,  ISBN ----, e-ISBN (PDF) ----

Engineering Risk Management Thierry Meyer, Genserik Reniers,  ISBN ----, e-ISBN (PDF) ----

Timothy Havranek, Doug MacNair

Multicriteria Decision Making Systems Modeling, Risk Assessment, and Financial Analysis for Technical Projects

Authors Timothy Havranek MBA, PMP [email protected] Doug MacNair PhD [email protected]

ISBN 978-3-11-076564-9 e-ISBN (PDF) 978-3-11-076586-1 e-ISBN (EPUB) 978-3-11-076590-8 Library of Congress Control Number: 2022950902 Bibliographic information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available on the Internet at http://dnb.dnb.de. © 2023 Walter de Gruyter GmbH, Berlin/Boston Cover image: iStock/Getty Images Plus Typesetting: Integra Software Services Pvt. Ltd. Printing and binding: CPI books GmbH, Leck www.degruyter.com

To my wife Margret Havranek and my stepdaughter Pamela Joy Hogue. Thank you for your love, inspiration, and strength. Timothy Havranek To Jessica, Carolyn, and Sean. The three best things that ever happened to me. Doug MacNair

Acknowledgements I would like to express special thanks to the following individuals who played significant roles in the development of this book. Jamie Z. Carlson who assisted me in obtaining copyright permissions and with overall administrative support. Jamie stepped in to provide this help, just when I needed it most. Tamara Underiner, for her assistance in editing original text and providing overall writing encouragement. Also, thank you Tamara for your many years of invaluable friendship. Christopher Carlson, who stepped in, along with his wife Jamie, to provide needed technical review. Chris has been a longtime friend and fellow career traveler in the environmental consulting industry. Mica Hanish, who assisted in developing the MCDM template and provided much needed assistance in developing the conjoint survey worksheets based on Taguchi design of experiments. Mica did this work as an intern and during her senior year at Allegheny College where she graduated with a bachelor’s degree in mathematics and a minor in economics. Lastly, but certainly not least, I would like to thank my coauthor Doug MacNair, PhD, for joining me on this project. Doug helped to expand the decision analysis methods I was using when we first met over twenty years ago and together, we have had the opportunity to employ them on many projects. In addition, Doug has been my long-term economics mentor. Timothy Havranek I want to thank Tim for inviting me to participate in writing this book. We’ve worked together for many years on many MCDM projects and talked many times about writing this book. But if it weren’t for Tim’s passion about the value of decision analysis and his desire to document our experience, the book would have remained just talk. More importantly, I want to acknowledge learning, friendship, and fun I have had collaborating with him. I also want to give a shout-out to Sean MacNair for his contributions in proofing and editing the final versions of the book. It is not easy to be a recent college graduate in philosophy and to help an engineer and economist get a book across the finish line, but he did a great job. Doug MacNair

https://doi.org/10.1515/9783110765861-202

Contents Acknowledgements List of Figures

XV

List of Tables

XVII

1 1.1 1.2

VII

Introduction 1 Our Complex World 1 Is a Structured Decision Process Really Necessary? Common Objections 2 1.2.1 It’s My Job to Make Good Decisions 3 1.2.2 I Can’t Trust a Computer Program to Make Decisions for Me 3 1.2.3 It’s Possible to Make Decision Models Reach Any Conclusion That You Want 3 1.2.3.1 Advocacy-Based Approach and Potential Effect of Cognitive Biases 4 1.2.3.2 Inquiry-Based Approach 5 1.2.4 Garbage In, Garbage Out 6 1.2.5 We Have Our Own Standardized Decision-Making Process 6 1.3 Why This Book? 7 1.4 MCDM Applications 8 1.4.1 Energy Planning and Policy 8 1.4.2 Oil and Gas Exploration and Production (E&P) 8 1.4.3 Healthcare Decision Making 9 1.4.4 Environmental Management 10 1.5 Terminology 10 1.5.1 Multicriteria Decision Analysis (MCDA) 11 1.5.2 Decision 12 1.5.3 Decision Analysis 12 1.5.4 Good Decision 12 1.5.5 Uncertainties 13 1.5.6 Risk 13 1.5.7 Decision Makers 13 1.5.8 Stakeholders 13 1.5.9 Values 13 1.5.10 Objectives 14 1.5.11 Goals 14 1.5.12 Criteria 14 1.5.13 Objectives Hierarchy 14 1.5.14 Alternatives 15

X

1.5.15 1.5.16 1.5.17 1.6 1.6.1 1.6.2 1.6.3 1.6.4 1.6.5 1.6.6 1.6.7 1.6.8

2 2.1 2.2 2.3 2.4 2.5 2.6 2.6.1 2.6.2 2.6.3 2.6.3.1 2.6.3.2 2.6.3.3 2.6.3.4 2.6.3.5 2.7

3 3.1 3.2 3.2.1 3.2.2 3.2.3 3.2.4 3.2.5

Contents

Level or Score 15 Trade-Offs 15 Optimization 16 Benefits of MCDM 16 Includes Nonfinancial as well as Financial Objectives 16 Offers Insights in Values and Trade-Offs 17 Identifies Creative and Implementable Alternatives 18 Reveals the Impact of Uncertainties 18 Identifies the Alternative Most Aligned with Decision Makers’/ Stakeholders’ Values 20 Provides a Singular Comprehensive Analytical Framework 20 Communicates the Totality of Consequences 21 Potential for Significant Cost Savings 21 References 23 Introduction of a Case Study 26 Integrated Capital Assessments 26 History of Town of Greenville 29 Developing a Strategic Plan for Greenville 31 Community Interests 31 Purpose of Proposed MCDM 32 Superfund Cleanup 33 Sediment Project Status 34 Remedial Costs 35 Impact of Sediment Remediation Alternatives on Greenville Strategic Plan 36 Alternative 1 – Complete Dredging 36 Alternative 2 – Hotspot Dredging 38 Capping 39 Monitored Natural Attention 39 Confined Disposal Facility (CDF) 39 Example Survey 40 References 40 Foundations of MCDM 41 Reviewing the Fundamentals: A Strategy for Simplifying MCDM Fundamental Elements of Decision Problems 42 Choices 43 Known Facts 45 Chance Events 45 Constraints 48 Value Measures 50

41

Contents

3.2.6 3.3 3.4 3.4.1 3.4.2 3.5 3.5.1 3.5.2 3.5.3 3.5.3.1 3.5.3.2 3.5.3.3 3.5.4 3.5.5 3.5.6 3.5.7 3.6 3.6.1 3.6.2 3.6.2.1 3.6.2.2 3.6.2.3 3.6.2.4 3.7 3.7.1 3.7.2 3.7.3 3.7.4 3.7.5 3.8 3.8.1 3.8.2 3.8.3 3.8.4 3.8.5 3.9 3.9.1 3.9.2 3.9.3 3.9.4

Preferences 50 Fundamental Concepts 50 Systems Engineering and Systems Thinking 51 Systems Modeling 51 Systems Thinking 52 Fundamental Concepts of Probability Theory 55 Bayes’ Formula and Subjective Probabilities 61 Describing Experimental Data 65 Graphical Representations of Experimental Data 65 Frequency Histograms 65 Discrete and Continuous Distributions 67 Cumulative Distribution Functions – Discrete and Continuous Distributions 69 Measures of Central Tendency 70 Measures of Dispersion 72 Distinguishing Properties of Probability Distributions 73 Probability Distributions Most Useful for MCDM 75 Fundamental Concepts of Finance 77 Time Value of Money 77 Net Present Value 77 Nominal Versus Real Dollars 79 Real Discount Rate 80 Advantages of Stepwise Structuring of Cash Flow Analysis Within Spreadsheets 80 Calculating NPV 81 Fundamental Concepts of Economics 82 The Basic Economic Problem 83 Opportunity Cost 83 Rational Person Assumption 84 Revealed Preference Analysis 84 Stated Preference Analysis 84 Behavioral Economics 85 Risk Aversion in Gains 86 Risk-Seeking in Losses 87 Choice Preference as a Function of Decision Frame 88 Cognitive Biases 90 Emotions and Rationality 92 Decision Quality 93 Appropriate Frame 94 Creative Doable Alternatives 95 Meaningful Reliable Information 95 Clear Values and Trade-Offs 95

XI

XII

3.9.5 3.9.6

4 4.1 4.2

Contents

Logical Correct Reasoning 95 Commitment to Action 95 References 96

The MCDM Process 98 Is the Complete MCDM Process Required for Every Decision? 99 How Much Effort Should Be Invested in the Decision Analysis Process? 101 4.3 Outline of the MCDM Process 101 4.4 Inviting Stakeholders to Share in Decision Making 103 4.4.1 Potential Levels of Stakeholder Involvement 103 4.4.2 Recommended Levels of Stakeholder Involvement 104 4.5 Structure Phase 105 4.5.1 Concept of the Decision Hierarchy 106 4.5.2 The Participants in the Decision Process and Their Roles 107 4.5.2.1 Decision Executive 107 4.5.2.2 Decision Analysis Facilitators 108 4.5.2.3 Decision Review Board 109 4.5.2.4 Project Team Members 109 4.5.2.5 Stakeholders 109 4.5.2.6 Subject-Matter Experts 110 4.5.3 Preframing Meeting Activities and Exercises 110 4.5.4 Framing Meeting Exercises 112 4.5.4.1 Background Information Review 113 4.5.4.2 Stakeholder Analysis and Engagement 115 4.5.4.3 Document Policies 117 4.5.4.4 Develop Objectives Hierarchy 117 4.5.4.4.1 Top-Down Objectives Hierarchy Approach 117 4.5.4.4.2 Bottom-Up Objectives Hierarchy Approach 118 4.5.4.4.3 Blended Objectives Hierarchy Approach 119 4.5.4.5 Example Objectives Hierarchy 119 4.5.4.6 Identifying Value Measures (Criteria) 121 4.5.4.6.1 Double Counting 121 4.5.4.6.2 Conceptual Independence 121 4.5.4.6.3 Stated in Natural Units 122 4.5.4.6.4 Categorical Criteria 122 4.5.4.6.5 Identifying a Starting List of Criteria 122 4.5.4.7 Designing Alternatives 123 4.6 Exercises 125 References 125

Contents

5 5.1 5.2 5.2.1 5.2.2 5.2.3 5.2.4 5.2.5 5.2.6 5.3 5.3.1 5.3.2 5.4 5.4.1 5.4.2 5.4.3 5.5 5.5.1 5.5.2 5.5.3 5.6 5.6.1 5.6.2 5.6.3 5.7 5.8 5.9

The Evaluation Process – Building the MCDM Model 127 Quantifying Preferences and Uncertainties 127 Conjoint Surveys 128 Administering the Conjoint Survey 129 Example Conjoint Survey 130 Design of Experiments 132 Evaluating Conjoint Surveys Using Linear Regression 134 Calculating Criteria Weights 135 Interpreting Linear Regression Results 135 Fitting PDFs to Actual Data 144 Using @Risk’s Distribution Fitting Feature 144 Selecting Which Fitted Distributions to Use 149 Defining Input Distributions Based on Expert Judgment 155 Class Estimating Exercise 156 Documenting Expert Elicitation Results 161 Shaping PDFs Based on Subject-Matter Expert Elicitation 162 Estimating the Probability of Discrete Events 163 The Probability Wheel 164 Standardized Probability Phrases and Tabular Visual Aids 165 References to Processes Where Probabilities Are Well Known 166 Structuring the MCDM Model 167 The Additive Value Function 167 Probabilistic Normalization 168 Examples MCDM Model Structure – Conceptual and Actual 168 MCDM Template 172 Cash Flow Model Template 173 Exercise 180 References 180

6 6.1 6.2 6.2.1 6.2.2 6.2.3 6.2.4 6.3 6.4 6.4.1

The Agreement Phase 182 Addressing the Issue of Distributed Authority 183 Case Study of Stakeholder Involvement 184 Consult Level 184 Involve Level 185 Collaborate Level 185 Empower Level of Stakeholder 186 Developing Output Results 186 Communicating Insights 189 Output Cumulative Distribution Functions and Probability Distributions 190 Sensitivity Tornado Diagrams 192 MCDM Score Stacked Bar Graph 192

6.4.2 6.4.3

XIII

XIV

6.4.4 6.4.5 6.4.6 6.5 6.6

Contents

Comparing Alternative Risks Using Box and Whisker Plots Comparison of Value Measures Across Alternatives 195 Output Descriptive Statistics 195 Commit to Implement 196 Summary Statement 196

Appendix A Example Stakeholder Survey

199

Appendix B Case Study: Example Stakeholder Analysis Appendix C Case Study Objectives Hierarchy Appendix D Case Study Strategy Table

203

205

207

Appendix E Case Study: Completed Conjoint Surveys and Objectives hierarchy 211 Index

221

194

List of Figures Figure 1.1 Figure 1.2 Figure 1.3 Figure 1.4 Figure 1.5 Figure 1.6 Figure 1.7 Figure 2.1 Figure 2.2 Figure 2.3 Figure 2.4 Figure 3.1 Figure 3.2 Figure 3.3 Figure 3.4 Figure 3.5 Figure 3.6 Figure 3.7 Figure 3.8 Figure 3.9 Figure 3.10 Figure 3.11 Figure 3.12 Figure 3.13 Figure 3.14 Figure 3.15 Figure 3.16 Figure 3.17 Figure 3.18 Figure 3.19 Figure 3.20 Figure 3.21 Figure 3.22 Figure 3.23 Figure 4.1 Figure 4.2 Figure 4.3 Figure 4.4 Figure 4.5 Figure 4.6 Figure 5.1 Figure 5.2 Figure 5.3 Figure 5.4 Figure 5.5 Figure 5.6

Growth of MCDM applications in the environmental field 11 MCDA publications by environmental application area during 2000–2010 11 Simplified objectives hierarchy 15 Example PDF representing time to removal of fish consumption advisories 19 Comparative cumulative distribution functions 19 Conceptual model MCDM as a singular comprehensive analytical framework 21 Comparison of value measures across alternatives 22 Definition of the four capitals: Capitals Coalition, “Why a Capitals Approach” 26 Principles for undertaking capital assessments: Capitals Coalition 27 Key definitions for capital assessments 28 Port of Greenville’s Superfund site 30 Green River shoreline development strategic choices 44 Venn–Euler diagram for Green River strategic decision 44 Venn–Euler diagram for a new alternative 45 The random variable X as a probability tree 46 Tree diagram for Green River dredging costs 47 Probability distribution – Green River dredging cost 48 Systems identification and relevant events 54 Polyhedral representation of system structure 54 Example of probability tree for air travel involving a connecting flight 58 Example Probability Tree for Air Travel Involving Conditional Probabilities 60 Frequency histogram for an experiment involving 10,000 rolls of a pair of dice 66 Histogram of Phase B investigation cost data 66 Fitted distribution Phase B investigation costs 68 Application of @Risk’s Define Distribution Truncate Setting 69 Cumulative distribution function for the outcomes of a pair of dice 70 Cumulative distribution function for Phase B investigation costs 71 Cost probability distributions for competing project strategies 72 Screenshot of @Risk Define Distribution Feature 75 Risk neutral coin toss wager 86 Increased reward of coin toss wager 87 Decision tree regarding competing loss choices 88 Decision tree for alternate framing example 90 The decision quality chain 94 A suggested prescription for resolving decisions 100 Outline of the MCDM process 101 The decision hierarchy 107 Framing meeting exercises 112 Example objectives hierarchy 120 Example strategy table 124 Criteria weights from Table 5.2 conjoint survey 135 Objective hierarchy with criteria weights included 143 @Risk fit distributions to data window 146 Fit distributions to data, distributions tab 147 @Risk distribution fitting ranking methods 148 Comparison of Phase B cost data with Weibull cumulative distribution function 150

https://doi.org/10.1515/9783110765861-204

XVI

Figure 5.7 Figure 5.8 Figure 5.9 Figure 5.10 Figure 5.11 Figure 5.12 Figure 5.13 Figure 5.14 Figure 6.1 Figure 6.2 Figure 6.3 Figure 6.4 Figure 6.5 Figure 6.6 Figure 6.7 Figure 6.8

List of Figures

Comparison of Phase B cost data with Weibull cumulative distribution function 151 P–P plot of Phase B investigation cost versus fitted Weibull distribution Q–Q plot of Phase B investigation cost data versus fitted Weibull distribution 154 Binomial (8, 90%) distribution 160 Binomial (8, 37.5%) distribution 161 Example probability wheel 164 Visual aid in estimating probabilities 166 Example project criteria weights 172 @Risk simulation settings, sampling tab 188 @Risk simulation settings, general tab 188 MCDM score cumulative distribution functions 190 MCDM score probability distributions 192 Example sensitivity tornado diagram 193 MCDM stacked bar graph 193 Box and whisker diagram for comparing alternative risks 194 In river cleanup duration alternative comparison 195

154

List of Tables Table 2.1 Table 2.2 Table 3.1 Table 3.2 Table 3.3 Table 5.1 Table 5.2 Table 5.3 Table 5.4 Table 5.5 Table 5.6 Table 5.7 Table 5.8 Table 5.9 Table 5.10 Table 5.11 Table 5.12 Table 5.13 Table 5.14 Table 5.15 Table 5.16 Table 5.17 Table 5.18 Table 5.19 Table 6.1

Sediment remediation costs 36 Sediment remediation direct impacts 37 Possible outcomes for a roll of a pair of dice 55 Probabilities of rolling the numbers 2 through 12 56 Probability distributions most useful for MCDM 75 Example conjoint survey criteria definitions 131 Example conjoint survey alternatives scoring table 132 Regression statistics produced by MS Excel LINEST function 136 Output regression statistics 137 Summary of value measure weights and tests for significance 140 Stakeholder conjoint survey – primary concern annual green energy to community 141 Stakeholder’s weights and significance results 141 Phase B cost data 145 Tabular comparison of input data to Weibull distribution 152 Expert elicitation documentation – cost input parameter 162 Conceptual summary of MCDM approach 169 Example actual project criteria 170 Example actual project non-normalized scores 170 Example actual project normalized scores 171 Example actual project MCDM score 171 Cash flow model input table cost portion 175 Cash flow model, timing of cost elements 177 Cash flow model, cost distribution 179 Cash flow model, net present value determination 180 Alternative present value cost descriptive statistics 196

https://doi.org/10.1515/9783110765861-205

Access to MCDM Modeling Template and Case Study Solution Throughout this book, we refer to the MCDM Template that readers can use for developing their own MCDM models. This template, along with output results for the Chapter 2 Case study, can be found at the following link. https://www.degruyter.com/document/isbn/9783110765861/html Statement Regarding Lumivero and Palisade Corporation We refer many times to the Microsoft Excel add-in programs @Risk and PrecisionTree. These programs were originally produced by Palisade Corporation. Palisade Corporation was acquired by Lumivero while this book was being written. In the future, this may have an impact on the references that readers can use to obtain help and information regarding these software programs. However, as of this writing these links are still active and can be found at the Palisade Corporation’s website https:// www.palisade.com. Should this change in the future, readers are referred to Lumivero’s website https://lumivero.com. Palisade Corporation’s (Now Lumivero’s) Decision Tools Suite Purchasers of this book may use the following link to download a special Textbook Edition of the Decision Tools Suite Industrial, which includes @Risk, Precision Tree, TopRank, NeuralTools, StatTools, Evolver, and RiskOptimizer. It will expire one year after installation. www.palisade.com/bookdownloads/degruyter

https://doi.org/10.1515/9783110765861-206

1 Introduction Multicriteria decision making (MCDM) is a structured collaborative process and analytical framework for making complex, high-stakes decisions involving competing objectives, multiple stakeholders, and significant uncertainties. It’s a powerful tool that helps decision makers identify creative, high value strategies for all types of technical projects and business initiatives. Most importantly, MCDM increases the likelihood of achieving intended outcomes. MCDM can be used to markedly improve the decision-making process in any organization including public corporations, private businesses, and governmental entities. When used by public corporations and private businesses, MCDM helps identify strategies that not only increase the likelihood of achieving financial objectives but also of achieving other nonfinancial objectives valuable to the company and its stakeholders. When used by government entities, MCDM helps identify innovative strategies for improving infrastructure, increasing efficiency in delivering public services, and enhancing recreational spaces, while increasing transparency and incorporating the public voice. Ultimately, MCDM can be used by government entities to create more sustainable societies.

1.1 Our Complex World Although there are many reasons why businesses and government entities may choose to employ MCDM, perhaps the best is that it provides a structured process for making sound decisions in an increasingly complex, interconnected, and uncertain world. There are numerous sources of these complexities, interconnections, and uncertainties (hereafter referred to as complicating factors); some examples include: – economic globalization, – changing laws and regulations (domestic and international), – climate change effects (e.g., water scarcity, draughts, wildfires, and severe weather), – worldwide pandemics, – social media impacts (both positive and negative), – environmental and community activism, – advances in information technology, – changes in consumer values, and more recently, – increased investor, and consumer, interest in how well companies perform regarding a host of environmental, social, and governance (ESG) metrics. These, and other complicating factors, interact in ways that can affect both business and government by: https://doi.org/10.1515/9783110765861-001

2

– –

– – –

1 Introduction

creating supply chain disruptions; changing demands for – manufactured products, – natural resources, and – labor skills; increasing inflation and price instability; creating stock market volatility; and causing increases in unemployment in some areas and industries and labor shortages in others.

Given these complicating factors and their effects, corporate managers, private business owners, and government officials can no longer afford to make decisions based simply on costs and expected financial returns. They must now consider a host of other objectives including improving ESG metrics, and increasing sustainability, resiliency, and stakeholder satisfaction. In addition, business managers must ensure that their decisions are fully aligned with their company’s vision, mission, and values. Such alignment helps maintain investor confidence, customer base, and brand image. For government officials, this alignment helps increase overall satisfaction with public services. Lastly, to be truly successful, these managers and officials must assess the risks that might prevent them from achieving their intended objectives (even when making high-quality decisions) and take proactive measures to mitigate/manage such risks during implementation.

1.2 Is a Structured Decision Process Really Necessary? Common Objections Many leaders, business executives, and government officials would likely agree that our world is becoming increasingly complex. Also, given this complexity, it’s likely that many would agree that making high-quality decisions has never been more important. However, as practicing decision analysts, we are aware that some leaders and managers, when presented with the concept of a structured decision process, will raise objections by making statements such as: – “It’s my job to make good decisions.” – “I can’t trust a computer program to make decisions for me.” – “It’s possible to make decision models reach any conclusion that you want.” – “Garbage in, garbage out.” – “We have our own standardized decision-making process.” In many ways, it’s understandable that a number of leaders, managers, and officials would raise objections and make statements such as those presented earlier. Therefore, it’s important that anyone attempting to promote decision analysis within their

1.2 Is a Structured Decision Process Really Necessary? Common Objections

3

organization be prepared to respond to such statements. To assist with this preparation, each of the earlier statements is addressed in the following sections.

1.2.1 It’s My Job to Make Good Decisions When the concept of a structured decision process such as MCDM is first presented to leaders, business executives, and government officials, some may feel that the promoters of the decision process are suggesting that these individuals have been making poor decisions. This is not the case. Any successful business, organization, leader, executive, or manager has achieved their success by having a track record of making good decisions. However, the question is not one of good decisions versus bad decisions, but of good decisions versus even better or higher value decisions.

1.2.2 I Can’t Trust a Computer Program to Make Decisions for Me One of the myths associated with MCDM (or any decision process that involves the use of computer modeling and analysis) is that it requires the decision makers to blindly accept the results of analysis, i.e., that decision analysis and computer modeling will somehow replace the decision maker or undermine his or her value or authority. Rather than forcing decision makers to choose a particular alternative, MCDM provides the decision makers with new insights and information for making more informed decisions. MCDM does not replace decision makers; it simply provides them with information regarding the likely outcomes associated with the various choices, thereby enabling them to make higher value decisions.

1.2.3 It’s Possible to Make Decision Models Reach Any Conclusion That You Want This statement is often made by those highly skeptical of decision models, or any form of data analytics for that matter. Such individuals recognize that by changing the weights of various decision criteria (indication of stakeholder preferences) as well as other important input parameters such as criteria scores, capital expenditures, operating expenses, sales volume, and product prices, one could make a particular alternative look more valuable than others. Although it’s possible that an individual or group could manipulate a decision model by changing the input parameters until their preferred alternative outranks the others, a reasonable question is “what would be the motivation?” As we will see in Chapter 4, the MCDM process is designed to create a collaborative journey of inquiry with the destination of finding the best, highest value alternative. If the model is artificially manipulated to produce a predetermined result, this essentially undermines the

4

1 Introduction

whole point of MCDM, or any other decision support methodology for that matter. The decision makers and facilitators would instead be engaging in an advocacy-based approach rather than the MCDM inquiry-based approach to decision making. 1.2.3.1 Advocacy-Based Approach and Potential Effect of Cognitive Biases The terms “advocacy-based approach” and “inquiry-based decision making” are well defined by David C. Skinner in his book, Introduction to Decision Analysis. According to Skinner, the traditional decision-making process in most organizations is an advocacy-based approach. This process involves someone in authority stating a problem to be solved or a project to be evaluated. Then a person or a team goes away and gathers data, picks an alternative, performs an evaluation and presents a recommendation to the decision maker. If the results of the analysis are consistent with the decision maker’s beliefs and preferences, the recommendation is approved and funded. If the recommendation does not match the decision maker’s beliefs or preferences, the team is sent to rework this evaluation. This cycle can be repeated several times, as the decision maker may not agree with a person or team’s analysis of the situation, their proposed decision, the assumptions, or the analysis which led to the business case recommendation [1].

Skinner refers to this process as an “advocacy-based approach” because it is like that of a lawyer presenting a case to a judge. In this analogy, the person or project team represents the lawyer while the decision maker represents the judge. Note that in the traditional advocacy-based approach, the decision maker is in a sense persuading the project team to adjust their assumptions, forecasted costs and benefits, values, and ultimately alternatives until they reach the decision maker’s preconceived notion of the correct decision. In many cases, this may be acceptable since, as previously stated, successful business leaders, managers, and government officials achieved their positions by having a record of making good decisions. It’s likely that many of their decisions were made using the traditional advocacy-based approach. However, this approach, when applied to high-stakes decisions involving significant uncertainties in our complex world, will, at best, result in decisions that are merely sufficient, rather than providing the highest possible value. At the other end of the spectrum, it is possible that the advocacy-based approach can lead to decisions that increase the likelihood of unfortunate and unintended outcomes. If we refer back to the objection that “decision models can be adjusted to reach any conclusion that you want,” we can see that this is exactly what happens with the traditional advocacy-based decision-making process. The fact that the team performing the analysis may be persuaded by the decision maker to manipulate the decision inputs does not indicate that decision modeling is faulty. Rather, it indicates the team and decision makers are attempting to revert back to their traditional advocacy-based approach. One might wonder: why would a business executive, project team, or any other stakeholder in the decision-making process seek to adjust inputs or information in

1.2 Is a Structured Decision Process Really Necessary? Common Objections

5

order to reach a preconceived result? In most cases, the goal is not to intentionally mislead; rather, such individuals simply believe that, given their experience, training, and understanding, they know what’s best. Therefore, they are attempting to ensure that the weight of evidence points to their alternative choice and validates their belief. This is fine if they are indeed correct. However, it’s possible that they are engaging in a cognitive bias of one type or another. A cognitive bias is a systematic error in thinking that occurs when people are processing and interpreting information in the world around them, and affects the decisions and judgments that they make [2]. According to Kendra Cherry, the human brain, although powerful, is subject to limitations. Cognitive biases are often a result of our brain’s attempt to simplify information processing. They work as rules of thumb that help us make sense of the world and reach decisions with relative speed [3]: The concept of cognitive bias was first introduced by researchers Amos Tversky and Daniel Kahneman in 1972. Since then, researchers have described a number of different types of biases that affect decision-making in a wide range of areas including social behavior, cognition, behavioral economics, education, management, healthcare, business, and finance [4].

Wikipedia’s List of Cognitive Biases indicates that at the time of this writing there are 188 known cognitive biases [5]. A visual representation of all 188 cognitive biases as an infographic is available from Design Hacks Company [6]. Some of the more common types of cognitive biases include: – Confirmation bias – the tendency to listen more often to information that confirms our own beliefs and ignoring or discounting information that is counter to our beliefs – Anchoring bias – the tendency to be overinfluenced by the first piece of information that we hear – Availability heuristic – the tendency to overestimate the probability of something happening based on an event that readily comes to mind – Optimism bias – the tendency to overestimate the likelihood that good things will happen In reviewing the traditional decision-making process as described by Skinner, one can easily imagine how any of these four common biases, or any of the other cognitive biases, could influence the advocacy-based approach. 1.2.3.2 Inquiry-Based Approach According to Skinner, Following an inquiry-based approach requires keeping an open mind and looking to develop alternatives and options which maximize value to the organization. In an inquiry-based approach, the decision maker must validate and accept key process outputs before moving to the next phase in the process. By doing so, the whole team (decision maker and analysis team) develops a

6

1 Introduction

shared understanding of the problem and is able to explore the where and why value is created in the various alternatives. When it is time to make the decision, there is no advocating for a position – the whole team understands the value proposition and is ready and excited to pursue the course of action [7].

MCDM makes use of an inquiry-based approach. However, Skinner describes a process whereby the decision maker and analysis team work for the same organization. The MCDM approach as presented here reaches beyond a singular organization to include the values and preferences of stakeholders that exist both within and outside of the organization. Nevertheless, the goal is still to create a shared understanding of the issues, increase transparency, and ultimately to achieve agreement with, or at least acceptance of, a preferred course of action.

1.2.4 Garbage In, Garbage Out This is still a statement that one hears from time to time whenever the concept of computer modeling or any type of data analysis comes up. In a way, this objection is at the other end of the spectrum from the concern that, by altering inputs, it’s possible to make a decision model say anything you want. In that case, the concern is that the analysis team would intentionally replace valid inputs with those more suited to their liking. In the case of garbage in, garbage out, the concern is that low quality or inaccurate data will be used in the modeling effort. Whenever a well-conducted MCDM is performed, this concern is unfounded. The inputs used for MCDM, as in other analytical methods, are gathered by qualified scientists, engineers, and other subject matter experts working in conjunction with decision analysis facilitators focused on identifying and removing cognitive biases. There is no reason to simply assume that such data will not be accurate or representative. Another important feature of the MCDM process is that it allows for sensitivity analysis, which is a way of determining the degree of impact that each input parameter has on output parameters of interest, such as the multicriteria score for each alternative. There are some parameters that have very little impact on the overall score. As such, it is not important that these input parameters be perfectly representative. Therefore, additional efforts to validate these inputs would not be necessary. On the other hand, the sensitivity analysis typically indicates that certain input parameters significantly affect the outputs of interest. Efforts should be made to ensure that such input parameters are as representative as possible.

1.2.5 We Have Our Own Standardized Decision-Making Process Many organizations have developed standardized decision processes with the goal of ensuring decision quality. These organizations understand that high-quality decisions

1.3 Why This Book?

7

are important to the competitiveness and overall survival of their organization. The MCDM methods presented in this book are not a replacement to such methods. Rather, MCDM is offered as a tool to support the understanding of issues, a way of informing debate, and the political process leading to the ultimate decision [8].

1.3 Why This Book? There are many fine books on decision analysis including those focused on single objective (or single criterion), which is typically used for evaluating competing investment opportunities in the financial realm, as well as MCDM. Some of the books are ground breaking, like Ralph L. Keeney’s and Howard Raiffa’s Decisions with Multiple Objectives. Others provide a great introduction to decision analysis, such as Making Hard Decisions by Robert T. Clemen and Introduction to Decision Analysis by David C. Skinner. Some function as great academic text books such as Foundations of Decision Analysis by Ron A. Howard and Ali E. Abbas. The goal of this book is to write from the perspective of a practitioner. As such, the focus is not on expanding the frontiers of decision science such as identifying new alternative ranking algorithms, criteria weighting techniques, or evaluating the merits of competing methodologies. Rather the focus is on informing, simplifying, and providing practical tools for use by practitioners and others seeking ways of introducing decision analysis into their organizations. This book takes a pragmatic business and economics view toward evaluating competing investment alternatives and/or capital project strategies. It provides a practical step-by-step process for using a structured decision analysis framework to evaluate, understand, quantify, and measure project strategies in light of a multitude of objectives and success criteria. This process helps stakeholders (internal and external) achieve a shared understanding of project issues and facilitates convergence toward a mutually acceptable solution. The approach considers available choices, identified uncertainties, constraints, necessary trade-offs, and preferences to identify solutions that maximize overall benefits while minimizing costs and risk. Advances in computer technology allow for investment strategies to be evaluated against multiple criteria within one integrated platform. This book guides the reader in performing multicriteria decision modeling (MCDM), including the use of Monte Carlo simulation, within an MS Excel environment using native MS Excel and Lumivero’s (formerly Palisade Corporation’s) Decision Tools suite. Example model structures, screen shots, formulas, and output results are provided throughout the book using illustrative case studies.

8

1 Introduction

1.4 MCDM Applications Within the world of business and government, the number of potential MCDM applications is quite large. This section reviews some of the areas where MCDM has been successfully applied. The examples presented are not intended as a complete listing of potential applications. Rather, they serve to demonstrate the range of MCDM applicability in hopes of inspiring the reader to seek out other applications within their sphere of influence. Industries where MCDM has been successfully employed include: – Energy planning and policy – Oil and gas exploration and production (E&P) – Healthcare decision making – Environmental management

1.4.1 Energy Planning and Policy The use of MCDM in energy planning and policy has been occurring since the early 1970s. Some of the early uses include the siting of new electric power-generating facilities and transmission lines. In Energy Decision and the Environment, Benjamin F. Hobbs and Peiter Meier note that energy planning and policy applications number in the hundreds [9]. These authors provide a representative sampling of energy applications that includes environmental impact assessment, transmission system design, expansion of power generation capabilities, and energy planning for developing countries.

1.4.2 Oil and Gas Exploration and Production (E&P) The petroleum industry was an early adopter of formalized quantitative decision analysis methods largely due to the high-stakes nature of oil and gas exploration, development, and production. These projects require large capital investments (often in the hundreds of millions or even billions of dollars) and involve numerous risks and uncertainties associated with factors such as: – whether exploratory wells will be successful; – whether new production wells will perform as expected; – actual cost of new production facilities (offshore and onshore); and – commodity price of oil and gas at the time of production. The term “formalized quantitative decision analysis” was used to describe the oil and gas industry’s early use of decision support methods, rather than MCDM. This is because early applications by this industry were focused on the single criterion of maximizing expected net present value (NPV), also known as expected monetary value.

1.4 MCDM Applications

9

These applications made use of decision trees and/or Monte Carlo simulation to perform probabilistic financial modeling of competing alternatives. Such methods, focused on a single criterion, are best referred to as quantitative decision analysis. Research by Eleni Strantzali and Konstantinos Aravossis confirms that single criterion approaches have historically dominated decision making in the oil and gas sector [10]. “However, given the complexity and conflicting interests of involved actors in the decision-making process, the use of multicriteria evaluation techniques is gaining momentum,” especially in the upstream sector of the oil and gas industry [11]. This sector includes five developmental phases ranging from exploration and development through production, life extension and, ultimately, abandonment. Mahmood Shafiee, Isaac Animah, Babakalli Alkali, and David Baglee note that decision support methods such as MCDM have received the most attention during the development stage, followed by the production and exploration stages [12].

1.4.3 Healthcare Decision Making In 2014, the International Society for Pharmacoeconomics and Outcomes Research established the MCDA emerging Good Practices Task Force [13]. Note that MCDA and MCDM are interchangeable terms (see Section 1.5 on terminology). This task force was charged with establishing a common definition for MCDA and developing good guidelines for conducting MCDA in healthcare decision making. In their initial report, this group provided examples of the use of MCDA in different kinds of healthcare decision making: – Benefit–risk assessment (BRA): This is a methodology used by regulatory agencies for balancing the multiple benefits and risks of medical products for the purpose of informing regulatory decisions. The European Medicines Agency BenefitRisk Project developed and tested methods for performing BRA. One of the results of this study is that the project suggested that a full MCDA model would be most useful for difficult or contentious cases, when the benefit–risk balance is marginal and could tip either way depending on the judgments of the clinical relevance of the effects, favorable and unfavorable, and in the case of many conflicting attributes [14]. – Health technology assessment (HTA): This is the systematic evaluation of properties, effects, and/or impacts of healthcare technology. HTA should include medical, social, ethical, and economic dimensions, and its main purpose is to inform decision making in the health area. These assessments look at benefits and efficacy, clinical and technical safety, and cost-effectiveness. Informed decision making comprises issues surrounding coverage and reimbursement, pricing decisions, clinical guidelines and protocols, and lastly, medical device regulation. The main purpose of HTA is to inform a policy decision making in healthcare, and thus improve the uptake of cost-effective new technologies and prevent the uptake of technologies that are of doubtful value for the health system (Pan

10

–

–

–

1 Introduction

American Health Organization, 2021). MCDA has been used by HTA bodies located in Germany, Thailand, and Italy [15]. Portfolio decision analysis in a pharmaceutical company: MCDA was used by Allergan for prioritizing projects on the basis of value for the money. MCDA was used to collapse multiple benefits into a single risk-adjusted benefit. “The study concluded that the MCDA process helps to increase communication across silos, to develop a shared understanding of the portfolio as a whole, and the transparency makes it easy to brief upwards, and provides an audit trail of the decisionmaking process” [16]. Local commissioning – a local healthcare planner in the English National Health System: The Isle of Wight Primary Care Trust System used MCDA to support the allocation of resources across 21 interventions in 5 priority health areas. The resulting plan was approved by the Isle of Wight Primary Care Trust Board. “The study concluded that MCDA has the potential to support local health planners in their task of allocating fixed budget to a wide range of types of health care” [17]. Shared decision making – evaluating cancer screening alternatives: The analytical hierarchy procedure (AHP), a form of MCDA, was used to elicit decision priorities of people with average risk of colorectal cancer at four primary care practices located in the United States. The study concluded that patients were able to perform AHP analysis and that it was possible to use these techniques in patient-centered decision making.

1.4.4 Environmental Management A study performed by Ivy B. Huang, Jeffrey Keisler, and Igor Linkov indicates a significant increase in MCDA applications in the environmental field [18]. Figure 1.1 shows an exponential growth rate in environmental MCDA applications for the time period from 1990 through 2010. This graph was generated using data from table 5 presented in Huang, Keisler, and Linkov. A breakout of publications by the application type for the years 2000 through 2010 was prepared by Huang, Keisler, and Linkov and was summarized in table 2 of their report. Figure 1.2 summarizes the information contained in table 2 of Huang, Keisler, and Linkov.

1.5 Terminology There are many terms that are used interchangeably, or defined differently, by individuals working in various fields associated with decision analysis. For purposes of clarity, the definitions and meanings for a number of MCDM terms are provided in this section.

1.5 Terminology

11

200

Number of MCDA Papers

180 160

y = 2E-162e0.188x R2 = 0.9156

140 120 100 80 60

40 20 0 1990

1995

2000

2005

2010

Year Figure 1.1: Growth of MCDM applications in the environmental field.

Air Quality Emissions Natural Resources Mgmt. Remediation / Restoration Quality Management Sustainable Manufact. Waste Management Spatial GIS Stakeholders Energy Assement Env. Impact Strategy 0

10

20 30 40 Number of Publications

50

60

Figure 1.2: MCDA publications by environmental application area during 2000–2010.

The terms are presented in logical order/chronological order as they might be considered by those entering a decision analysis process.

1.5.1 Multicriteria Decision Analysis (MCDA) The term “multicriteria decision analysis” is used synonymously with MCDM. We prefer MCDM because the purpose of the decision analysis process is ultimately to decide on an alternative, not just analyze how well alternatives perform regarding a set of criteria. Also, in many cases, MCDA does not include the use of uncertain inputs represented

12

1 Introduction

by probability distribution functions (PDFs) and does not make use of Monte Carlo simulation (this is not true in all cases but is very common). This book is focused on stochastic MCDM which involves the use of probabilistic inputs and Monte Carlo simulation. In the definition of criteria (Section 1.5.12), we note that a better name for this term would be value measures, because criteria are used to measure how well objectives are being met and ultimately, we prefer certain objectives because they are consistent with our values. Lastly, some authors use the term “multiobjective decision making” (MODM) which is a very descriptive term since we seek to make decisions that will increase the likelihood of achieving our objectives. However, MCDM is used more commonly than MODM and therefore for this book we chose to stay with MCDM.

1.5.2 Decision “A decision is an irrevocable allocation of resources; irrevocable in the sense that it is impossible or extremely costly to change back to the situation that existed before making the decision” [19]. This definition assumes that the decision is not merely a thought process, but an actual commitment to a course of action.

1.5.3 Decision Analysis “Decision analysis is a philosophy and a social-technical process to create value for decision makers and stakeholders facing difficult decisions involving multiple stakeholders, multiple (possibly conflicting) objectives, complex alternatives, important uncertainties and significant consequences” [20]. This general definition covers both single objective (i.e., single criterion decision analysis) and MCDM.

1.5.4 Good Decision “A good decision is one that is logically consistent with our preferences for potential outcomes, our alternatives, and our assessment of uncertainties” [21]. Note that this definition does not include a statement about the actual outcome of a decision. The decision analysis literature has long recognized that there is a distinction between a good decision and a good outcome (see R.A. Howard [19]). A good outcome is what we hope will happen. It is possible for a good decision to have a bad outcome and for a bad decision to have a good outcome. These possibilities exist because of the uncertainties (i.e., lack of information) that exist at the time our decisions are made. However, it is generally accepted that good decisions are the best (and perhaps only) way of increasing the likelihood of good outcomes.

1.5 Terminology

13

1.5.5 Uncertainties Uncertainties exist either because of lack of information – i.e., we don’t have enough information to make exact estimates about the future – or because we are involved in a truly probabilistic process such as the flip of an unbiased coin.

1.5.6 Risk “Risk is an uncertain event or condition that, if it occurs, has a positive or negative effect on one or more project objectives” [22]. It is the presence of risk that can cause a good decision to have a bad outcome or vice versa.

1.5.7 Decision Makers People invested with the authority and responsibility to make decisions for an organization or enterprise [20].

1.5.8 Stakeholders Stakeholders are individuals or organizations that are directly or indirectly affected by project outcomes and results, whether positive or negative. The list of stakeholders includes those individuals and organizations that merely believe they will be affected by a project, whether or not such beliefs are justified. Each project has its own unique set of stakeholders. Later, we will see in our discussion of criteria and criteria weights; it is the values and preferences of the stakeholders that determine criteria weights.

1.5.9 Values According to Ralph A. Keeney: Values are principles for evaluation. We use them to evaluate the actual or potential consequences of action and inaction, of proposed alternatives, and of decisions. They range from ethical principles that must be upheld to guidelines for preferences among choices. We make them explicit through statements expressing value judgements. To render value judgments useful for decision making we must be precise about their meaning. We can articulate this meaning qualitatively by stating objectives, if desirable, we can embellish it with quantitative value judgement. Ethics, desired traits, characteristics of consequences that matter, guidelines for action, priorities, value tradeoffs, and attitudes toward risk all indicate values [23].

14

1 Introduction

1.5.10 Objectives “An objective generally indicates “the direction” in which we should strive to do better” [24]. Objectives typically include words such as minimize, maximize, increase, or reduce. Objectives are derived from values. The process of going from values to well-defined objectives requires some rather hard thinking, or what Keeney refers to as value-focused thinking. There are two primary types of objectives: fundamental objectives and means objectives. Fundamental objectives are what we ultimately care about in a decision. Means objectives describe how the fundamental objectives will be achieved.

1.5.11 Goals Goals specify a level of achievement to strive toward. They are either achieved or not. “However, within our subject matter [decision analysis] objectives are more relevant for evaluating strategic decision problems” [25]. Although for this reason the use of the term “goals” is avoided within this book, the distinction is worth mentioning here.

1.5.12 Criteria Within the decision analysis literature, the term “criteria” (or the singular criterion) is used interchangeably with a number of other terms, including attributes [26], value measures [27], evaluation measures [28], metrics [29], and parameters [30]. Although the names differ, a review of the literature indicates that these terms can be defined as a measuring scale indicating the degree of attainment of an objective [31]. The preferred term is value measure because it is most descriptive. However, the term criteria is well embedded in the decision analysis literature and within the name MCDM. Therefore, the terms criteria and value measures are used interchangeably within this book. The reader should assume they have the same meaning.

1.5.13 Objectives Hierarchy An objectives hierarchy is a tree-like structure that relates fundamental objectives, means objectives, and value measures. Creating the objectives hierarchy is an important first step in the decision analysis process. Figure 1.3 presents a simplified objectives hierarchy.

1.5 Terminology

15

Fundmental Objective

Means

Means

Objective

Objective

Value Measure

Value Measure

Value Measure

Figure 1.3: Simplified objectives hierarchy.

1.5.14 Alternatives “Alternatives are what you can do – a feasible allocation of resources which are available now or can become available to the decision maker(s)” [32]. It is worth noting that during the planning stage of nearly every technical project, it’s seldom the case that there is only one strategic decision to make. The more common situation is that there are a number of different strategic decisions that must be made about various aspects of a technical project. Each strategic decision contains its own finite set of available choices. As we shall see later in the discussion of strategy table, alternatives are a collection of strategic choices.

1.5.15 Level or Score “The specific numerical rating for a particular alternative with respect to a specified evaluation measure [i.e., value measure or criteria] constitutes its level (score)” [33].

1.5.16 Trade-Offs Trade-offs involve giving up a little of something valued to gain more of something valued higher. There is seldom, if ever, an ideal alternative that perfectly achieves all decision maker and stakeholder objectives. The typical case is that the available alternatives meet the objectives to various degrees. Thus, the decision makers/stakeholders are forced to make trade-offs. Such trade-offs indicate the decision makers’/stakeholders’ preferences.

16

1 Introduction

1.5.17 Optimization Optimization means efficiently using available resources to achieve best possible outcomes, given the constraints of time, money, energy, technology, and societal preferences. Optimization often requires making tough trade-offs on how to best use resources given various system constraints.

1.6 Benefits of MCDM “The principal aim [of MCDM] is to help decision makers learn about the problem situation, about their own and other’s values and judgements, and through organization, synthesis, and appropriate presentation of information to guide them in identifying . . . a preferred course of action” [34]. As much of the published research on MCDM makes clear, its primary benefits are that it: – includes nonfinancial as well as financial objectives, – provides insights into values and trade-offs, – identifies creative and implementable alternatives, – reveals the impact of uncertainties, – identifies the alternative most aligned with decision makers’/stakeholders’ values, – increases transparency, – provides a singular comprehensive analytic framework, and – has the potential for significant cost savings.

1.6.1 Includes Nonfinancial as well as Financial Objectives MCDM includes nonfinancial objectives and seeks value measures that are aligned with the objectives that are stated in ways meaningful to decision makers/stakeholders. For example, greenhouse gas emissions might be used as one of the value measures for evaluating the sustainability of various alternatives for a given technical project. The natural unit of measure for this value measure is tons of carbon dioxide (CO2). However, a more meaningful unit of measure for communicating greenhouse gas impacts to nontechnical project stakeholders might be equivalent household emissions of CO2. A fair question regarding nonfinancial objectives and value measures is why not simply convert them into dollar equivalents and rank alternatives based on the net additive value of their benefits and costs – i.e., use cost–benefit analysis (CBA)? Although CBA is a valid approach, it can be challenging to communicate to external stakeholders. It requires the stakeholders to trust that the methods used by analysts to convert nonfinancial criteria into dollar equivalents properly account for stakeholder values and preferences. MCDM also explicitly considers the distribution of benefits over different stakeholder groups.

1.6 Benefits of MCDM

17

Where there are clear financial stakes involved, MCDM has distinct advantages as well. MCDM is an extension of single objective (or single criterion) decision analysis typically used for evaluating competing investment opportunities in the financial realm. NPV is the criterion used in such applications. But where NPV is seen as a single criterion, uncertain future costs and revenues result in the introduction of a new criterion, a.k.a. “risk.” This criterion is typically measured in standard deviation (usually symbolized by the Greek letter sigma, σ). Therefore, single objective decision making is actually multi-objective (i.e., multicriteria) decision making with the dual objectives of maximizing NPV and minimizing risk (i.e., standard deviation). Clearly, it is extremely difficult, if not impossible, to identify any decision that truly involves only one objective. Financial decision analysis (as well as MCDM) uses Monte Carlo simulation, or other probabilistic methods such as decision tree analysis or influence diagrams to account for uncertainties. In the financial realm, the simulation outputs of interest are the mean NPV and standard deviation. When MCDM is applied to technical projects, the outputs of interest include the MCDM score, NPV, and present value (PV) cost. Note that NPV assumes a revenue as well as a cost stream; many projects, such as those involving environmental remediation, involve only costs, so PV cost is more appropriate in such cases. Monte Carlo simulation provides a host of descriptive statistics pertaining to outputs of interest including measures of central tendency (mean, median, and mode) and of dispersion (variance, standard deviation, range, and probability percentiles). It also provides a number of output graphs including PDFs, cumulative distribution functions (CDFs), and sensitivity tornado diagrams that, taken together, provide insights into the risks associated with each alternative.

1.6.2 Offers Insights in Values and Trade-Offs Many wrongly assume that MCDM provides an objective analysis and thus relieves decision makers of the responsibility of making difficult decisions. This assumption is far from accurate. The truth is that all decisions involve subjectivity, and this subjectivity is not reduced by the use of MCDM. Valerie Belton and Theodore J. Stewart provide one of the best expressions for the purpose of MCDM, and the role played by subjectivity and value judgment: MCDA [i.e., MCDM] is an aid to decision making, a process that seeks to: – Integrate objective measurement with value judgment; – Make explicit and manage subjectivity Subjectivity is inherent in all decision making, in particular in the choice of criteria [value measures] on which to base the decision and the relative “weight” given to those criteria. MCDA does not dispel that subjectivity; it simply seeks to make the need for subjective judgments explicit and the process by which they are taken into account transparent [35].

18

1 Introduction

The success of the MCDM process depends in large part on identifying a set of criteria (i.e., value measures) that decision makers/stakeholders can agree upon or at least accept. There will always be a certain degree of subjectivity in the selection of criteria. Later, in Chapter 4 (see Section 4.5.4.6), recommendations are provided for identifying, defining, and establishing units of measure and weighting various criteria.

1.6.3 Identifies Creative and Implementable Alternatives “Focusing on the values that should be guiding the decision situation makes the search for new alternatives a creative and productive exercise. It removes the anchor of narrowly defined alternatives and allows clear progress toward solving the problem” [36]. The idea of focusing on values first is what Keeney refers to as valuefocused thinking and is contrasted with alternative-focused thinking. When faced with a difficult decision, it seems tempting to begin with a set of alternatives that is comfortingly narrow, but often restrictively so. The focus on values first, and the conversion of values into fundamental and means objectives, often leads to the identification of new and creative alternatives that otherwise would have been overlooked.

1.6.4 Reveals the Impact of Uncertainties Perhaps the greatest benefit of MCDM is that it reveals the impacts of uncertainties. This is accomplished by first acknowledging that uncertainties exist and work to replace the point estimates regarding the criteria scores of various alternatives with PDFs. For example, in the context of environmental remediation involving sediment contamination, assume that one of the means objectives for the community acceptance criterion is the time until removal of fish consumption advisories, measured in years. The actual number of years remaining until the removal of fish advisories cannot be known with certainty. However, expert judgment and/or statistical analysis can be used to establish PDFs for each alternative (i.e., establish alternative scores) such as that shown in Figure 1.4. This PDF informs the decision makers/stakeholders that the mean time for the removal of the fish advisory is 17 years with a 90% confidence interval of 13–23 years. In addition to revealing the impact of uncertainties on the performance of alternatives with respect to value measures, MCDM reveals the impact of uncertainties on the overall performance of each alternative as measured by the multicriteria score. The higher the multicriteria score, the greater the alignment of an alternative with decision makers’/stakeholders’ values. Figure 1.5 presents CDFs of the score for three competing alternatives produced via Monte Carlo simulation. Alternative A is located furthest to the right and is more vertical in nature. This indicates that Alternative A best meets decision makers’/stakeholders’ objectives since

19

1.6 Benefits of MCDM

13

23

5.0%

8%

90%

5.0%

7%

Probability

6% 5% 4% 3%

1%

+1 SD = 20

-1 SD = 14

Mean = 17

2%

0% 10

12

14

16

18

20

22

24

26

28

30

32

Years until Removal Figure 1.4: Example PDF representing time to removal of fish consumption advisories.

100%

Cumulative Probability

90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 10

20

30

40

50

MCDM Score Alt. A

Alt. B

Figure 1.5: Comparative cumulative distribution functions.

Alt. C

60

70

20

1 Introduction

it has the highest score and the least amount of risk. Alternative A outperforms the others throughout the range of probability and, therefore, is considered stochastically superior. In such a case, the decision makers/stakeholders can be confident that Alternative A is their best possible alternative regardless of the underlying uncertainties regarding the future performance of various value measures. Had the curves crossed, additional sensitivity analysis could be used to determine which value measures are driving the crossings. At that point, a decision could be made about whether additional data should be gathered (value of information analysis) to reduce uncertainty, or whether the uncertainty (risk) is acceptable. The equations used to calculate the multicriteria scores and the method used to produce these curves are described in Chapter 5 (Section 5.6).

1.6.5 Identifies the Alternative Most Aligned with Decision Makers’/Stakeholders’ Values MCDM makes use of an objective function and probabilistic methods to identify the alternatives most aligned with decision makers’ and stakeholders’ values. The results may indicate that the alternative most aligned with the decision makers’ values is different from the one most aligned with stakeholders’ values. This is because the decision makers and the stakeholders have different trade-off preferences which are reflected in their respective criteria weights. A difference in the preferred alternative is not a negative result. In such cases, sensitivity analysis can be used to determine which of the evaluation measures and/or criteria weights are leading to the preferred alternatives. This information, having been made more explicit, can be used for communication, understanding, and negotiation among and between the various decision maker and stakeholder groups.

1.6.6 Provides a Singular Comprehensive Analytical Framework The ability to act as a singular comprehensive analytical framework that integrates other frameworks is a powerful benefit of MCDM, perhaps on par with revealing insights into uncertainties. MCDM can include the results from human health-based risk assessment (HHRA), ecological risk assessment (ERA), life cycle analysis (LCA), CBA, environmental footprint analysis, cost effectiveness analysis, sustainability analysis, natural resource damage assessment (NRDA), or any other framework available now or in the future. According to Magnus Sparrevik, David N. Barton, Mathew E. Bates and Igor Linkov: Multicriteria decision analysis has advantages over lower dimension decision methods such as CBA [cost benefit analysis] and CEA [cost effectiveness analysis] because it can simultaneously incorporate stakeholder values for different aspects (criteria) of the decision and allows for ranking among the alternatives that incorporates criteria measured on different scales (e.g. including both monetary and non-monetary aspects) [37].

1.6 Benefits of MCDM

21

Figure 1.6 presents MCDM as a singular comprehensive analytical framework by including the results of other analytical frameworks in this case including HHRA, ERA and life cycle analysis.

ERA HHRA HH LCA

MCDM

Figure 1.6: Conceptual model MCDM as a singular comprehensive analytical framework.

1.6.7 Communicates the Totality of Consequences MCDM communicates the totality of consequences in three ways. The first is that it quantifies (i.e., scores) all value measures so they may be compared across alternatives. Figure 1.7 presents a comparison of three possible value measures that could be associated with a sediment Comprehensive Environmental Response Compensation and Liability Act (CERCLA, also known as “Superfund”) site. Note that this graph indicates the mean scores of various value measures. Within the Monte Carlo simulation model, the value measure scores are represented by PDFs. The second way is that it rolls up the totality of consequences in the form of an objective function, i.e., the MCDM score. The third way, which has already been discussed, is that it reveals the impacts of uncertainties. Without a tool such as MCDM to quantify and evaluate the totality of consequences, the selection of a final remedy can seem arbitrarily focused on a singular objective such as removing the greatest volume of impacted material, regardless of ancillary consequences.

1.6.8 Potential for Significant Cost Savings The potential cost savings associated with decision analysis in general are well known. Parnell et al. (2013) reported that the benefits-to-cost ratio associated with investing in better decisions is frequently on the order of a thousand to one. Such returns can be attributed to identifying new and better alternatives, avoiding costly risks, and efficiently reaching high-quality decisions. The authors’ experience in assisting with the

22

1 Introduction

application of decision analysis to remediation projects has been somewhat similar, with savings more on the order of hundreds to one. By informing debate and influencing the political process such that alternatives other than the most costly and aggressive are agreed upon and accepted, the savings associated with applying MCDM to any type of technical project can be substantial. Volume of Contaminated Sediment Removed Cubic Yads - Millions

2.5 2.0 1.5 1.0 0.5 0.0 A

B C Alternative

D

E

Truck Traffic - 1,000s

Total Truck Traffic 100 90 80 70 60 50 40 30 20 10 0 A

B

C Alternative

D

E

Years

Habitat Restoration Time 45 40 35 30 25 20 15 10 5 0 A

B

C Alternative

D

Figure 1.7: Comparison of value measures across alternatives.

E

References

23

Now that we have introduced the motivation for using MCDM, described its benefits, and defined a number of MCDM terms, we are ready to describe the MCDM process and how to implement its various phases and steps. We begin in Chapter 2 by introducing a fictitious case study that is used to demonstrate various components of the MCDM process. In addition to the case study, we also use examples and outputs from actual MCDM projects to further illustrate MCDM process steps, inputs, and outputs. In order to protect client confidentiality regarding such examples, we do not mention the client’s name, site or project name, or location of the site (in terms of city or state, we do mention the general geographic area for context). In addition, to further protect confidentiality, we’ve altered elements of these projects, while retaining the relative relationships of various project issues (e.g., the relative cost ranges associated with alternatives). Chapter 3 provides an overview of the fundamental concepts from various fields of study that are employed by the MCDM process and are useful for both the process and outcome results. We’ve found over the years that it is a lack of familiarity with some of these concepts, or perhaps more accurately, the amount of time since many professionals studied these concepts, that causes MCDM to seem much more complex than it actually is. As we suggest at the beginning of Chapter 3, those familiar with these fundamental concepts can choose to skip Chapter 3 or simply review those concepts where they need to refamiliarize themselves. Chapter 4 provides a complete overview of the MCDM process including describing its three phases (i.e., structure, evaluation, and agreement) with their associated process steps. In addition, Chapter 4 covers the structure phase of the MCDM process including framing meeting exercises to assist in creating a shared understanding of issues among decision makers and stakeholders. This chapter also includes discussion of stakeholder levels of involvement and engagement in the MCDM process for the purpose of increasing agreement upon and acceptance of selected project alternatives. Chapter 5 covers the evaluation phase of the MCDM process. This phase is focused on steps for constructing the MCDM model including quantifying preferences, developing probabilistic inputs, and structuring the model to relate inputs to outputs. Chapter 6 focuses on the agreement phase which includes a review of model results and their interpretation and ultimately on selecting a high-value project alternative.

References [1] [2] [3] [4]

Skinner, D. C. (1999). Introduction to decision analysis: A practitioners guide to improving decision quality, Second Edition. Gainesville, FL, USA, Probabilistic Publishing 1999, p. 4. Cherry, K. What Is Cognitive Bias. (Accessed from verywellmind, January 10, 2020 at https://www. verywellmind.com/what-is-a-cognitive-bias-2794963). Ibid. Ibid.

24

[5] [6] [7] [8] [9] [10] [11]

[12] [13]

[14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33]

1 Introduction

List of cognitive biases. December 11. (Accessed December 11, 2021 at https://en.wikipedia.org/wiki/ List_of_cognitive_biases). Design Hacks. (n.d.). cognitive-bias-codes-poster. Retrieved from www.designhacks.co:https://www.de signhacks.co/products/cognitive-bias-codex-poster. Skinner, D. C., Introduction to decision analysis: A practitioners guide to improving decision quality, Second Edition. Gainesville, FL, USA, Probabilistic Publishing 1999, pp. 6. French, S., & Argyris, N. (2018). Decision Analysis and the Political Process. Decision Analysis, 208–219. Hobbs B. F., Meier, P. Energy Decisions and the Environment. New York, NY, USA, Springer Science and Business Media, 2000. Strantzali, E., & Konstantinos, A. (2016). Decision making in renewable energy investments: A review. Renewable and Sustainable Energy Reviews, 885–889. Shafiee, M., Animah, I., Alkali, B., & Baglee, D. (2019). Decision support methods and application in the upstream oil and gas sector. Journal of Petroleum Science and Engineering, vol. 173, pp. 1173–1186 https://doi.org/10.1016/j.petrol.2018.10.050. Ibid. Marsh, K., Maarrten, I. Thokkala, P., et al. Multiple criteria decision analysis for health care decision making – an introduction: report 1 of the ISPOR MCDA emerging Good Practices Task Force. Value in Health 2016, 19 125–137. Ibid. p. 127. Ibid p. 127–130. Ibid p. 130. Ibid p. 130. Huang, I. B., Keisler, J., Linkov, I. Multi-criteria decision analysis in the environmental sciences: ten years of applications and trends. Science of the Total Environment, 2011, 409, 3578–3594. Howard, R. A. Decision analysis: applied decision theory. Proceedings of the Fourth International Conference on Operations Research, New York: Wiley-Interscience, 1966, pp. 55–71. Parnell, G. S., Bresnick, T. A., Tani, S. N., & Johnson, E. R., Handbook of Decision Analysis, John Wiley & Sons, Inc. Hoboken, NJ, USA, 2013, pp. 3. Ibid. pp. 93. Project Management Institute, A guide to the project management body of knowledge, sixth edition, Project Management Institute, Inc., Newtown Square, PA, USA, 2017, pp. 720. Keeney, R. A., Value focused thinking, A path to creative decision making, Harvard University Press, Cambridge, MA, USA, 1992, pp. 6–7. Keeney, R. L., Raiffa, H., Decisions with multiple objectives, Preferences and value tradeoffs. Cambridge University Press, Cambridge, MA, USA, 1992, pp. 6–7. Ibid. pp. 34. Keeney, R. A., Value focused thinking, A path to creative decision making, Harvard University Press, Cambridge, MA, USA, 1992. Parnell, G. S., Bresnick, T. A., Tani, S. N., & Johnson, E. R., Handbook of Decision Analysis, John Wiley & Sons, Inc. Hoboken, NJ, USA, 2013. Kirkwood, C. W. (1997). Strategic decision making, Brooks/Cole, Belmont, CA, USA, 1997, pp. 12. United States Environmental Protection Agency, Methodology for understanding and reducing a project’s environmental footprint, U.S. Environmental Protection Agency, Washington, DC, 2012. Holland, K. S., Lewis, R. E., Tipton, K., Karnis, S., Dona, C., Petrovskis, E., . . . Hook, C. (2011). Framework for Integrating Sustainability into Remediation Projects. Remediation Journal, 7–38. Kirkwood, C. W. (1997). Strategic decision making, Brooks/Cole, Belmont, CA, USA, 1997, pp. 12. Skinner, D. C., Introduction to decision analysis: A practitioners guide to improving decision quality, Second Edition. Gainesville, FL, USA, Probabilistic Publishing 1999, pp. 356. Kirkwood, C. W. (1997). Strategic decision making, Brooks/Cole, Belmont, CA, USA, 1997, pp. 12.

References

25

[34] Belton, V., & Stewart, T. J. Multi criteria decision analysis, An integrated approach, Kluwer Academic Publishers, Boston, MA, USA, 2002, pp.5. [35] Ibid, p. 3. [36] Keeney, R. A., Value focused thinking, A path to creative decision making, Harvard University Press, Cambridge, MA, USA, 1992, pp. 9. [37] Sparrevick, M., Barton, D. N., Bates, M. E., & Linkov, I. Use of stochastic multi-criteria decision analysis to support sustainable management of contaminated sediments, Environmental Science & Technology, 2011, 1326–4241.

2 Introduction of a Case Study The information presented in this case study is of a fictitious town, Greenville, located along the shoreline of one of the Great Lakes of North America (the US side). Although fictitious, this case study includes issues encountered by the authors while working as consultants in the areas of environmental remediation, restoration, and natural resource economics. As such, it represents an amalgamation of issues where MCDM can provide a value. By necessity, the case study is a simplified version of reality; however, we have worked to make the case study description significantly complex so that the difficulties in finding a comprehensive sustainable solution and the benefits of the MCDM process can be appreciated. Before turning to the specifics of the study, we introduce a globally recognized framework i.e., integrated capital assessment, which can be useful for MCDM assessments that involve multiple stakeholders, multiple programs, and multiple sources of value.

2.1 Integrated Capital Assessments A growing area where MCDM can provide value is in integrated capital assessments. Traditionally, capital has been thought of as the financial investments and productive assets of companies that create value to their owners and society. However, it is now recognized that natural, human, and social capitals are also key aspects of societal well-being as shown in Figure 2.1 [1]. If we invest in these forms of capitals, they will create value; if we degrade them, then our standard of living will not be sustainable. The capitals approach, or capital thinking, integrates a broad array of impacts into decision making.

Figure 2.1: Definition of the four capitals: Capitals Coalition, “Why a Capitals Approach”.

https://doi.org/10.1515/9783110765861-002

2.1 Integrated Capital Assessments

27

Many decisions by companies, communities, NGOs, and governments can have an impact on multiple types of capitals. And just as importantly, by carefully evaluating all of those impacts, stakeholders can understand and evaluate their dependency on these capitals. For example, increasing natural capital through greenways and urban forests can impact human capital by improving the health and well-being of local residents (and workers) and increase the produced capital by increasing worker productivity and employee retention. These links also serve to highlight the dependencies of businesses and workers on a sustainable stock of natural capital. Figure 2.2 summarizes the key principles for undertaking an integrated capital assessment [2]. It should come as no surprise that MCDM is one of the recommended approaches for conducting integrated capital assessments. Each of these principles, albeit with somewhat different language, underpins the MCDM approach. Therefore, our illustrative case study will be expressed in terms of an integrated capital assessment.

Figure 2.2: Principles for undertaking capital assessments: Capitals Coalition.

Figure 2.3 [3] summarizes a few concepts and terms from integrated capital assessments that are important considerations in MCDM because they affect the roles of internal and external stakeholders. As will be discussed in Chapter 4, the MCDM approach can be used in a wide variety of settings, with different levels of involvement by external stakeholders. In this case study, we consider the case where external stakeholders’ views will be explicitly considered in decision making, but they do not have a “binding” final vote in the selection of an alternative. Instead, their views on the relative importance of specific components or criteria will be solicited early in the process to help construct viable alternatives, from which the decision makers can select an alternative. This structure is a little bit different from other examples in the book; however, the methods for eliciting stakeholder values are the same. For clarity, we consider one

28

2 Introduction of a Case Study

strategic choice that might be part of an alternative to be evaluated by the MCDM model, i.e., the creation of riverwalks (see Figure 2.3).

Values Impacts

Outputs (physical results from combining the inputs, e.g. miles of trails)

(contribution, positive or negative, of the outputs on society, e.g. number of trail users, health impacts of hiking)

(Importance or worth of the output or impact, e.g. the weight given by stakeholders to providing trails, monetary value of health benefits)

Inputs (resources used in constructing the activity, e.g. material and labor for trails)

Figure 2.3: Key definitions for capital assessments.

Inputs and outputs are related to the technical specifications of the project activities. By technical specifications, there is a production function that transforms inputs to outputs for the activity, or project, under study, such as the creation of a riverwalk, or the technology to remove contaminated sediment, or the technology to produce a megawatt of renewable energy. How best to construct a riverwalk, in terms of materials, minimizing environmental impact, compliance with federal and state regulations, and the cost per mile, is a technical question that should by and large be addressed by engineers, scientists, and economists. Stakeholders, at large, should generally have a very limited role in determining the links between inputs and outputs because they do not have the expertise to contribute in addressing largely technical questions. We have seen many stakeholder workshops get derailed when stakeholders begin opining about the appropriate links about inputs and outputs (e.g., what type of dredge should be used mechanical or hydraulic). That doesn’t mean that there is only one way to achieve a given output or that the outputs from a given set of inputs are certain. Nor does it mean that stakeholders should take the word of technical people without question. The main point is that stakeholders provide more value to the process by focusing on impacts (value measures and value measure scores that may be achieved by various alternatives). In the integrated capitals approach, impacts and values describe how outputs affect stakeholders and society. Therefore, impacts as used in the capitals approach can be

2.2 History of Town of Greenville

29

thought of as the criteria or value measures used in MCDM and values can be thought of as the preferences that the stakeholders have for certain outcomes or impacts. In MCDM, stakeholders play the key role in evaluating impacts and values. Impacts and values are the appropriate domain of stakeholders, because they are the ones who will be affected by the project outputs or outcomes. When it comes to values, the opinions of engineers, ecologists, and economists are of little importance (unless they also happen to be local stakeholders). We have also seen stakeholder workshops get derailed when the technical folks spend too much time describing the How of a project at the expense of stakeholders discussing their views on the value of a project. We have also seen entire projects canceled or significantly delayed because the engineers and scientists involved with the project presumed that they understood what various stakeholders valued without actually speaking with them. There are numerous ways to elicit values. For the case study, we use conjoint analysis as described in Section 5.2. A key question then is, whose values should be elicited? And the answer will vary significantly based on the context. For example, the choice of who to include would be quite different if this project was being organized by government regulators to assess alternative remediation strategies, or by businesses interested in successfully implementing a particular economic development plan. In this case study, the project sponsors are the mayor and city council; therefore, they will determine the role of each stakeholder group in the evaluation process. The outline of the stakeholder engagement process we present in Chapter 4 could be used for any of these situations.

2.2 History of Town of Greenville The town of Greenville was founded in the late 1850s by Admiral Green. The town is economically depressed, but once it had a strong industrial base with a vibrant surrounding community. It is now struggling with the loss of its industries and the longterm environmental consequences of their activities. Initially, the town’s economy was focused on maritime transport and commercial fishing. In the early 1930s, the Port of Greenville was established with the installation of three slips capable of handling large cargo ships about 1 mile inland and along the west bank of Green River. Green River flows from south to north and ultimately empties into one of the Great Lakes. The primary materials moving through the Port in the 1930s were iron ore and coal. Later, in the 1940s, other materials handled at the Port included agricultural products, refined petroleum products, and manufactured goods. Greenville’s population peaked in the late 1960s at nearly 500,000. At this time, industries surrounding the town included a petroleum refinery, steel mill, large rail yard switching station, a coal-fired electric power generation plant, and a number of small manufacturing plants. Access to the interstate highway system occurred in the late 1950s.

30

2 Introduction of a Case Study

Greenville began experiencing economic decline in the late 1970s. The steel mill shut down in 1981 and the rail operations were cut back. Many of the manufacturing plants also moved or closed down. The town experienced a continuous decrease in population from the mid-1980s until the year 2000. At that time, the population stabilized. Many of the working residents of Greenville commute daily to a large city located approximately 25 miles west. Greenville, with a current population of 150,000, is situated primarily to the west of the Green River. However, the city limits also include areas located on the eastern bank of Green River (East Greenville, approximately 10 square miles). The population of East Greenville is approximately 20,000. Unemployment in East Greenville is nearly 20%, and most residents fall into low-income brackets. East Greenville derives its water supply from Green River. The old municipal water supply plant often has operation problems, and there are concerns that it fails to treat water adequately. Preteen children and teenagers from East Greenville often visit the east bank of Green River and may come into contact with contaminated sediments. Studies are being conducted to determine if residents of East Greenville have elevated levels of liver and other forms of cancer. The primary contaminants of concern associated with the Green River sediments include polychlorinated biphenyls (PCBs), mercury, and lead. A simplified map of the Greenville is shown in Figure 2.4. This figure shows that Green River divides the town into its east and west sections with the Port of Greenville

Port of Greenville Superfund Site Native American Nation Tribal Nation

Piper Plover Habitat Pristine Creek

East Greenville

Refinery

Port of Greenville

Green River Big River Industrial Canal

Rail Yard Power Plant

Figure 2.4: Port of Greenville’s Superfund site.

2.4 Community Interests

31

located on the western side of the Green River. A 5-mile Industrial Canal is located approximately 3 miles south of the Port of Greenville. The refinery is located just north of the canal, and the power plant and rail yard are located south of the Industrial Canal. Greenville has an opportunity to revitalize its economy and community. It is eligible for several federal and state redevelopment grants and loans, which has led to increasing interest from businesses. There may also be opportunities to develop renewable energy through wind turbines. Others believe that the riverfront area between Industrial Canal and the lake can be developed with a marina, restaurants, hotels, and recreational boating and fishing. Conversely, a Native American Nation and some community groups are keenly interested in environmental preservation and restoration. Decisions about the future have been delayed because of these divergent interests and because the community is awaiting the United States Environmental Protection Agency (USEPA) decision regarding cleanup of contaminated sediments in the Industrial Canal and Green River.

2.3 Developing a Strategic Plan for Greenville The lack of consensus about how to move forward has led to delays in decisionmaking and acrimonious meetings among stakeholders that seem to deepen divides in the community. The mayor and city council have decided to fund a study, “A 2030 Plan for a Sustainable Greenville,” which will use MCDM and an integrated capitals approach and develop a realistic future plan that meets the aspirations of local stakeholders. The plan will consider the benefits and costs of alternative future development scenarios and describe their impacts on the natural, social, and human capital of Greenville. The town recognizes that they do not have decision-making authority for all aspects of the plan. For example, they cannot decide the clean-up levels for the river cleanup, or authorize state development grants for industry or renewable energy grants. The goal of the plan is to develop a shared vision of community interests and provide a rigorous quantification of the basis for the plan, which can be shared with the community, and state and federal agencies

2.4 Community Interests Friends of the Green River is seeking environmental restoration of the river for purposes of nonmotorized boating, riverside biking, hiking, and eventually fishing. This group has publicly stated that it believes that complete dredging of all PCB sediments from the Green River and the Industrial Canal is the only acceptable way to address contaminated sediment. In addition, this group commissioned a study of the lakefront regarding piping plover habitat. The piping plover is a small sand-colored, sparrowsized shorebird that nests and feeds along coastal sand and gravel beaches in North

32

2 Introduction of a Case Study

America. This endangered species lays its eggs on open, pebbly beaches, making them vulnerable to predators and the loss of their habitat. Over the years, encroaching human development has reduced the number of nesting sites and contributed to the species’ decline. The study commissioned by Friends of the Green River indicates that lakefront area includes about 200 to 600 acres of potential piping plover habitat and nesting areas. The Friends of the Green want to develop a Habitat Conservation Plan that would protect all 600 acres of potential piping plover habitat. A local Native American Nation has a reservation about 10 miles east of Greenville. The reservation has a population of 2,400 and consists of approximately 20,000 acres. Much of this land is hardwood forest. Pristine Creek runs through the center of the reservation. Lake run and rainbow trout are present in this creek, and the reservation receives revenues through the sale of fishing licenses. However, a state-imposed fish consumption advisory of one meal per month due to PCB impacts has been established for these fish. It is believed that individuals living on the reservation exceed this limit on a regular basis. Some believe that these fish have been impacted as a result of PCBs in Green River and at the Port of Greenville. However, this has never been proven. Great Lakes Wind Power, Inc. has approached the Greenville mayor and city council in creating a wind farm within the lake about 1 mile north of Greenville. The proposal includes construction of twenty 300-feet tall wind turbines each capable of producing 2.75 MW of power. According to the US Energy Administration, the average US home uses 893 kWh of electricity per month [2]. “At a 42% capacity factor (i.e., the average among recently built wind turbines in the United States), the average turbine would generate over 843,000 kWh per month – enough for more than 940 average U.S. homes” [3]. Therefore, the proposed 20 towers could provide enough power for 18,800 US homes. Of course, the power could also support industrial and commercial users. In addition to approvals from the mayor and city council, permits from the State Environmental Department of Environmental Quality would be required to install the wind farm. The concept is supported by Grow Greenville, a business advocacy group, that believes that affordable renewable energy is the key to getting industry to move to Greenville. The installation of such a wind farm is expected to meet with strong opposition from the Friends of Green River, the Native American Nation and the Union of Concerned Developers, a group that believes lake and riverfront development for tourism, which is the key for Greenville’s future.

2.5 Purpose of Proposed MCDM The purpose of the MCDM process is to provide the mayor and city council with a small set of long-term strategic development alternatives that best reflect the residents’ vision for the future of Greenville. They recognize that the river cleanup (see Section 2.6) will have a significant impact on the future development options and will

2.6 Superfund Cleanup

33

be central to success of the MCDM. The city has approved a proposal for implementing an MCDM process with the following major steps: 1) Use an online survey to collect information about the natural, social, and human capital impacts that stakeholders believe should guide the evaluation of the alternatives. 2) Conduct a series of workshops with stakeholders to determine the weights of the potential project outcomes. 3) Form technical workgroups to develop a set of feasible alternatives (e.g., inputs and outputs) in each of the following component areas: a. Sediment cleanup b. Renewable energy and industrial development c. Tourism and outdoor recreation d. Environmental restoration e. Native American Nation impacts

2.6 Superfund Cleanup The most important topic to address in the strategic plan is the impact of the Superfund cleanup to address contaminated sediments in the Industrial Canal and the Green River. There is significant uncertainty about the timing, scope, impact, and cost of the cleanup. Moreover, it is critical to link the Superfund cleanup decisions to all of the components of the strategic plan in a systematic, integrated approach. Sediments in the Industrial Canal and in the Green River between the Industrial Canal and the lake have been undergoing environmental investigations since 2005. The site is being managed under the Comprehensive Environmental Response Compensation and Liability Act (CERCLA also known as Superfund) by the USEPA. The identified group of potentially responsible (PRPs) parties includes the power company, the petroleum refinery, the rail yard company, and the Port of Greenville (which is owned by the city). PCBs are the primary contaminants of concern. The PRP group submitted a CERCLA Remedial Investigation (RI) report during the fall of 2018. The report indicates that there are approximately 325,000 cubic yards of material having PCB concentrations greater than one part per million (1 ppm). This is the cleanup level most commonly selected by the USEPA for PCB-impacted sediment. Most of this material (250,000 cubic yards) is along a 1-mile length within the Industrial Canal and then along a 2-mile stretch from the entrance of the Industrial Canal moving northward to the lake. The highest concentrations of PCBs, in excess of 500 ppm, are seen within the finer sediments (silts and clays) of the Green River just downstream of the rail yard. The PRP group has internal disagreements over who is actually responsible for the PCB impacts in the canal and the Green River. They have agreed to cooperate and share costs equally for the RI, feasibility study (FS), remedial design, and remedial

34

2 Introduction of a Case Study

action (RA). After completion of the RA, they intend to negotiate proper allocation and, if necessary, litigate to achieve appropriate cost allocation. The power company is located furthest west on the south bank of the Industrial Canal and approximately 3 miles from the Green River. Historical reports indicate that the power company handled PCB materials in drums, underground storage tanks, and transformers onsite. Past site spills have been confirmed on site, and there is a PCB plume present in the groundwater. A free product plume (i.e., a separate liquid phase of PCB fluids) estimated at approximately 5,000 gallons has been discovered at the base of the uppermost aquifer and at a depth of approximately 15 feet. Groundwater flow is to the northeast (i.e., toward the Industrial Canal and Green River). The power company contends that the PCB plume at their site never reached the Industrial Canal. The refinery is situated 2 miles west of the Green River. The refinery claims to have never handled PCB materials of any type. However, historical records are limited and that statement cannot be verified. Former workers from the refinery have reported that they used to store PCB fluids in tanks and drums on site. There are significant soil and groundwater impacts at the refinery mostly involving diesel and gasoline range organic compounds. The rail yard is located at the confluence of the Industrial Canal and Green River. Historical records indicate that the former rail company, Fast Track, Inc., often conducted repairs of electric locomotives at the site in the 1960s through the 1970s, and may have released PCB fluids for the engines directly onto the rail yard. The rail yard was connected by storm drains to the Industrial Canal and the Green River. National Rail Company believes that it is not responsible for these impacts and purchased the site after a state-financed program removed PCB-impacted rail yard materials from the site. It is considering a legal attempt to be removed from the PRP group prior to implementation of the RA. Historical records show that the Port of Greenville handled drums containing PCB fluids in the past for use in hydraulic systems to operate large cranes.

2.6.1 Sediment Project Status As required by the CERCLA process, the PRP group founded the completion of a human health risk assessment (HHRA) and an ecological risk assessment (ERA) for the Greenville Superfund site. Both studies were completed in 2020. These studies indicate that removal or capping of PCP sediments having PCB concentrations in excess of 50 ppm would be protected of both human health and the environment. The PRP group is in the process of conducting the FS and expects to complete it in the Summer of 2023. The PRP group has worked to keep the local community informed via regular town hall meetings. Some of the PRP group representatives believe that involving the community may actually help in obtaining approval for lower cost RA alternatives such as capping and perhaps monitored natural attenuation for PCB sediments having

2.6 Superfund Cleanup

35

concentrations of less than 10 ppm. However, this opinion is not universally shared by all representatives involved in the PRP group. During these meetings, the PRP group has indicated that there are four primary RAs that have been identified to address sediment impacts: 1. Dredging, transportation, and disposal – This includes all sediments having PCB concentrations in excess of 1 ppm. 2. Hot spot dredging, transportation, and disposal – This includes all sediments having PCB concentration in excess of 50 ppm. This will involve about 100,000 cubic yards of material mostly in the last 2 miles of the Industrial Canal and within Green River. 3. Capping and MNA – This would require capping of approximately 110 acres of cap within the Industrial Canal and Green River. 4. Monitored natural attenuation. The mayor of Greenville, Rob Baron, a descendent of the founder of the Greenville Steel Mill, commissioned an independent study that suggests that the contaminated sediments should be placed within a confined disposal facility (CDF) constructed of the sheet pile material. This CDF would extend into the lake and create new land that could be used for constructing a gambling casino. Rob has suggested that the PRPs could pay for construction of the CDF and then donate the land to the city of Greenville. He has also suggested that the casino can pay for maintenance of the CDF once it has been installed. The planned casino which is expected to cost Casino Gaming International (CGI) more than $150 million to build would include more than 2,000 slot machines, buffets, a fine dining restaurant, and many other amenities. CGI estimates the annual revenues in excess of $125 million per year. This estimate assumes visitors of 1000–3000 people per day. Revenue share to the city of Greenville is estimated at $10 million annually. Lastly, Rob Baron claims the new casino would generate more than 300 full-time equivalent jobs. In the local newspaper, it has been reported that approximately 3% of casino visitors become compulsive gamblers, and the average compulsive gambler is $80,000 in debt.

2.6.2 Remedial Costs Preliminary estimates of the capital and monitoring costs for each of the five scenarios are provided in Table 2.1. Due to the many uncertainties associated with remediation cost estimation, these costs have been provided in the form of minimum, most likely, and maximum estimates (all costs are in millions of dollars). Information on the direct potential impacts associated with the remedial alternatives is summarized in Table 2.2. The USEPA is responsible for deciding which sediment remediation alternative will be employed at the site. Their decision will be based on the information provided in the

36

2 Introduction of a Case Study

Table 2.1: Sediment remediation costs. Alternative no.

Description

Capital cost

Annual operations and monitoring cost

Minimum Most Maximum Minimum Most Maximum likely likely 

Dredging, transportation, and offsite disposal of , cubic yards

$

$

$

N/A

N/A

N/A



Hot spot dredging, transportation, and disposal

$

$

$

N/A

N/A

N/A



Capping  acres

$

$

$

$.

$.

$.



Monitored natural attenuation

N/A

N/A

N/A

.

.

.



CDF

$

$

$

$.

$.

$.

FS being developed by the PRPs. The choice of a remedial alternative may have an impact on the timing and potential scope of the other components of the strategic plan. The decision reached by the EPA will be documented in their Record of Decision (ROD). This document is released for public comment and can be revised based on the comments received. The mayor and city council will have the opportunity to influence the USEPA’s decision during the public comment period. However, by keeping the USEPA informed of their strategic plan and by lobbying community support for the plan, they may be able to influence the USEPA prior to their release of the ROD. If the FS is completed in the summer of 2023, the ROD will likely be issued in the spring of 2024.

2.6.3 Impact of Sediment Remediation Alternatives on Greenville Strategic Plan The decision regarding the sediment remediation alternative will impact the Plan for Sustainable Greenville in many ways. Each of the alternatives and their potential impacts is discussed in the following section. 2.6.3.1 Alternative 1 – Complete Dredging If the complete Alternative 1 (dredging of all identified impacted sediment) is selected, then any plans to develop the riverfront between the Industrial Canal and lake with a marina, restaurants, hotels, and recreational boating and fishing would be delayed until after the dredging project is complete, which could be up to eight years. The

Description

Dredging, Transportation and Offsite Disposal of , cubic yards

Hot Spot Dredging, Transportation and Disposal

Capping  Acres

Monitored Natural Attenuation

CDF

Alternative No.











Table 2.2: Sediment remediation direct impacts.

,

N/A





,

Minimum

,

N/A





,

Most Likely

,

N/A





,

Maximum

CO and Other GHG Emissions (Standard Household Years)











Minimum











Most Likely











Maximum

Project Duration Working Seasons











Minimum











Most Likely











Maximum

Acres Piping Plover Habitat Destroyed

2.6 Superfund Cleanup

37

38

2 Introduction of a Case Study

earliest that riverfront development could begin is 2032. This would represent a significant delay to the city that is seeking to revitalize itself, through either wind power development or tourism. In addition, complete dredging will destroy all fishing habitat within the Green River for 10–20 years or more. However, Alternative 1 would likely be favored by the Native American Nation and Friends of the Green River since it achieves their stated goal of complete removal of contaminated sediments. Once the removal is complete, these groups would prefer to see the shorelines adjacent to the Green River developed as a scenic riverwalk including not only the preservation of existing habitat for migratory birds but the creation of new habitat as well. The trade-off between complete dredging and fish habitat has not been fully explored by the Native American Nation and Friends of the Green River. During dredging operations, which can only take place within the months of March through October (depending on weather conditions), the work will involve a significant amount of noise and light pollution (dredges operate on a 24-hour basis). In addition, there can be significant odors associated with the dredged material as it is brought to the surface. The prevailing wind direction is to the northeast which will impact those living in East Greenville. Another concern of regarding Alternative 1 is that it will extend far enough north on the Green River that it will impact the municipal water supply plant water intake location. This means that protective measures will have to be taken, not only in the form of silt curtains in the river but also the installation of new intake filtering equipment. Upgrades to the municipal supply plant to address this issue are expected to be somewhere within the range of 10–20 million dollars. 2.6.3.2 Alternative 2 – Hotspot Dredging The Hotspot Dredging Alternative could possibly be performed within the time span of 1 year with a maximum estimated time span of 4 years. This would allow earlier development of the riverfront if that were to become part of the city’s plan of a Sustainable Greenville. The results of the HHRA indicate that this alternative would be protective of human health. However, these results are not accepted by the Friends of the Green River, the Native American Nation, and many concerned citizens living in East Greenville. This alternative is protective of Piping Plover impact since the major hotspots (areas of highest PCB concentrations) along the Industrial Canal are approximately 0.5-mile upstream of the entrance of the Industrial Canal into the Green River. The areas of high PCB concentrations then extend for approximately 1.5-mile downstream within the Green River from the meeting point of the Industrial Canal. These higher concentrations exist more along the shoreline and within the fine-grained sediment. The areas of highest concentrations are along eastern shoreline of Green River just downstream of the Industrial Canals. This area is considered the most hazardous because this is where individuals may come into contact with this shoreline sediment. However, dredging of this sediment

2.6 Superfund Cleanup

39

will still present an impact to the intake to the municipal water treatment plant that will need to be addressed. This alternative may also result in the fish consumption advisories in place for a longer period of time. 2.6.3.3 Capping Sediment caps provide: – Physical isolation that prevents direct contact between impact sediment and biota – Stabilization that prevents resuspension and transport of sediments into other sites – Chemical isolation that prevents transport of dissolved contaminants within the water column Conventional cap designs involve multiple layers made up of sand, gravel, geotextile material, and nonpermeable layers such as high-density polyethylene. The areas where the cap would be installed include long section of the shorelines within the Industrial Canal and the Green River. Plans for installation of the cap includes restoration of the shoreline by planting native riparian vegetation. The riverfront area could begin being developed while the cap is installed. Both the HHRA and ERA indicate that once installed the cap is protective of human health and the environment. Environmental scientists estimate that the shoreline vegetation will be restored within 5 years. In addition, estimates are that fish habitat will be reestablished within 5–15 years. Because of the shoreline restoration, the capping alternative is conducive to a scenic riverwalk and recreational boating activities within the Green River. However, fish consumption advisories would remain in effect longer, which will reduce the attractiveness of the area as a tourist destination. The capping alternative has many environmental advantages; however, it is opposed by the Friends of the Green River and the Native American Nation who want the PCB contaminants out of the river. 2.6.3.4 Monitored Natural Attention Although monitored natural attenuation has been included as part of the FS, it is not expected that the USEPA would select this alternative. In addition, if selected, this alternative would meet with strong resistance from Friends of the Green River, the Native American Nation, and local concerned citizens. 2.6.3.5 Confined Disposal Facility (CDF) The mayor and some of the city council members are interested in the CDF alternative but it is not actually included in the FS. This alternative relies on the selection of Alternative 1 – complete dredging for the material to be deposited in the CDF. Talks between the mayor’s office and the PRPs indicate that if they were forced to implement

40

2 Introduction of a Case Study

Alternative 1, they would be willing to fund the placement of the dredged sediments within the CDF, but they would not pay for its construction. This cost would have to be paid by the town of Greenville or the CGI.

2.7 Example Survey An example survey that could be used to help identify criteria most important to various stakeholder groups has been included as Appendix A. The survey includes questions whereby participants can anonymously indicate their membership in a stakeholder group. The survey includes questions that the participants can use to register their feelings regarding various criteria that could be used to evaluate this case study. Lastly, survey includes open ended questions that enable the stakeholders to identify issues that may be of particular importance to them. The are a number of online survey tools available that could be used administer this survey. This survey example has been provided not only to illustrate the type of questions that could be used for this case study but also to assist readers with the creation of surveys for their own MCDM projects.

References [1] [2] [3]

Why a Capitals Approach? (Accessed on October 11, 2022). Why a Capitals Approach – The Capitals Coalition. Capitals Coalition, 2021. Principles of integrated capitals assessments (Accessed on October 11,2022) Principles-of-integrated-capitals-assessments_v362.pdf (capitalscoalition.org). Adapted from the Value Balancing Alliance.

3 Foundations of MCDM This chapter presents the fundamental elements that provide the input parameters forming the basis of any MCDM model and the fundamental concepts that provide the foundation of the MCDM process. As we will see in Chapter 4, a large portion of the decision framing process is focused on identifying, categorizing, and estimating these fundamental elements or input parameters. The fundamental concepts provide the means by which the input parameters are defined, perceived, and related. The fundamental concepts most important to the MCDM process are from the fields of mathematics, finance, economics, engineering, and the relatively new field of behavioral economics. The fundamental mathematical concepts derive from the subjects of probability theory, statistics, and to a small degree, calculus. Those concepts from the field of finance include time value of money and net present value (NPV). Fundamental concepts from the field of economics include opportunity cost, trade-off analysis, and revealed preferences. The study of systems engineering and systems modeling represents the primary engineering concepts. Behavioral economics is a relatively new field of study. It has its roots in the work of Israeli psychologists Amos Tversky and Daniel Kahneman on uncertainty and risk [1]. It combines elements of economics and psychology to understand why people make the choices that they do. In particular, it is concerned with understanding why human beings will not always make rational or optimal decisions, even if they have the information and tools available to do so [2]. The fundamental concepts from this field include cognitive biases, judgment under uncertainty, and the role that emotions play not only in detracting from but also in enhancing rational thinking. Recent discoveries indicate that rationality is dependent on emotion and that conversely a reduction in emotion (or even the removal of emotion which is often suggested in business analysis) may be a source of irrational behavior. These new discoveries regarding the role of emotions in decision making are based on the research of Antonio Damasio, a Portuguese-American neuroscientist who, at this writing, is the David Dornsife Chair in Neuroscience and professor of psychology, philosophy, and neurology at the University of Southern California [3].

3.1 Reviewing the Fundamentals: A Strategy for Simplifying MCDM When first exposed to MCDM, many assume that it is complex and difficult, and that effective use of the process requires high levels of mathematical and computer modeling skills. These are erroneous assumptions. What makes MCDM seem complex and difficult is that it combines concepts from so many fields of study. However, in nearly all cases, MCDM employs only the most basic concepts from each of the involved https://doi.org/10.1515/9783110765861-003

42

3 Foundations of MCDM

fields of study. This chapter reviews the basic concepts from the various fields of study as a strategy for simplifying MCDM. In terms of the mathematics involved, we hope to demonstrate that the mathematics is not overly difficult and that the hard work of model creation and execution is greatly simplified using Microsoft Excel (MS Excel) and the use of commercially available add-in programs, in particular, Lumivero’s @Risk software as well as other software contained within Lumivero’s Decision Tools Suite. Those readers familiar with fundamental concepts presented in this chapter may choose to proceed directly to Chapters 4–6 which deal with framing the decision problem, structuring the decision model, and interpreting the model results. However, such readers may still find it helpful to review this chapter as new insights may be gained regarding the application of these concepts to MCDM.

3.2 Fundamental Elements of Decision Problems An in-depth study of any subject often begins with breaking it down into its most fundamental elements. This is true whether subject is one of the natural sciences (e.g., chemistry, physics, biology, and geology) or one of the social sciences (e.g., economics, sociology, and psychology). This tradition of seeking the most fundamental elements of a subject dates back to the ancient Greek philosophers. For example, it was Democritus (circa 450 BC) who first postulated the concept of the atom as the most fundamental particle of nature (we now know that atoms are made of protons, electrons, and neutrons and that even more elemental particles known as quarks exist). Centuries of scientific research on the properties of atoms ultimately led to the periodic table of the elements, first developed by the Russian chemist Dmitri Mendeleev in 1869, that is widely used in chemistry, physics, and other natural sciences. Again, looking at fundamental elements, it was Euclid who, during the later part of the fourth-century BC, summarized what was known about geometry at that time. In his book, The Elements, Euclid summarized the axioms that define terms such as points, lines, and planes. These axioms, or self-evident statements, serve as the starting point for this field of study. In our study of MCDM, the most useful fundamental elements are: – Strategic choices – Known facts – Chance events – Constraints – Value measures – Preferences

3.2 Fundamental Elements of Decision Problems

43

In providing this list the authors are not suggesting that they are as powerful and profound as the periodic table or as groundbreaking as Euclid’s The Elements. We are suggesting that they are important components that are present in every decision problem. Also, we are suggesting that at nearly every point in the decision analysis process, it is useful to refocus on these elements and ask if all those that are appropriate and useful to the process have been identified and included. The list of elements is relatively short, as might be expected if we are indeed dealing with fundamentals. However, each one of these elements is a set, with each set containing many members. Although a significant number of definitions and terminology is provided in Chapter 1, additional definitions and discussion of each of these seven elements, as well as an expansion of previously provided definitions, are warranted here.

3.2.1 Choices In the discussion of the term alternative provided in Chapter 1, we described an alternative as a collection of strategic choices. Here our discussion of alternatives goes further to reveal how each strategic decision is a set of finite choices. To help visualize what is meant by there being a set of choices associated with each strategic decision, we will make use of the case study provided in Chapter 2 (hereafter, Case Study). In the Case Study, the decision makers have a number of strategic decisions to make regarding: – Green River shoreline development – Lake front development – Contaminated sediment cleanup The choices associated with the Green River shoreline development’s strategic decision include: – commercial development (marina, shop restaurants, and hotels) – industrial development, and – nature preserve/recreational development. A graphical depiction of the Green River shoreline development’s strategic decision and its associated choice set is presented as shown in Figure 3.1. This figure shows how this strategic decision appears when using Lumivero’s PrecisionTree program. Green River shoreline development’s strategic decision is just one of many strategic decisions that could be included in a decision tree used to model the Case Study, each with its own set of choice elements. In such a model, other strategic decisions are attached in the appropriate sequence to the endpoints (blue triangles) of previous decision nodes. As an alternative to using the decision tree format to represent strategic decisions, the choices associated with a particular strategic decision could be placed in a column

44

3 Foundations of MCDM

Commecial Development

Green River Shoreline Development

Greenville

Industrial Development

Nature Preserve

Figure 3.1: Green River shoreline development strategic choices.

under a heading containing the name of the decision. This approach is used when constructing a strategy table as presented in Chapter 4 (see Section 4.5.4.7). Last, to help reinforce the concept that strategic decisions represent a set of choices, we can represent them using symbolic notation and graphically using Venn–Euler diagrams. Symbolic notation for the Green River can be demonstrated by first denoting this strategic choice as C1. The choices within this set are the small letters a, b, and c to represent commercial development, industrial development, and nature preserve, respectively. Therefore, the symbolic representation of the Green River shoreline decision is presented in equation (3.1) and the graphical representation is presented in Figure 3.2: C1 = fa, b, cg

a

(3:1)

b

c C1

Figure 3.2: Venn–Euler diagram for Green River strategic decision.

The set notation and the Venn–Euler diagram for the Green River strategic decision is trivial at this point. However, thinking about strategic decisions in this way is helpful thinking in terms of systems and visualizing alternatives. For example, let C2 represent the Lake Front Development strategic decision and C3 represent the contaminated sediment cleanup strategic decision from our Case Study. Furthermore, let the letters {d, e, f, g} and {h, i, j} represent choices within the C2 and C3 strategic decision, respectively. Figure 3.3 shows that these three strategic choices are disjoint, meaning they share no common choices, and that a new alternative, denoted by A1, has been

3.2 Fundamental Elements of Decision Problems

45

created which is a set that includes one choice from each of the three strategic choices C1, C2, and C3, i.e., b, f, and h, respectively.

C2 e

d

g f

a

i

b h c

j C1

A1

C3

Figure 3.3: Venn–Euler diagram for a new alternative.

3.2.2 Known Facts Known facts are input parameters needed for the calculation of output values of interest such as the total MCDM score or the expected monetary value of each alternative. As their name implies, these parameters involve little or no uncertainty. Examples of known facts from the case study include: – acres of migratory bird habitat; – length in miles of the Green River shoreline from the industrial canal to the lake; – the base of the uppermost aquifer beneath the power company is 20 feet. It should also be noted that known facts can also be physical parameters such as the density of water or PCB fluids. Such physical parameters may be used to calculate intermediate parameters or value measures that ultimately feed into the MCDM score or expected monetary costs.

3.2.3 Chance Events Chance events are closely related to uncertainties and risk. Chapter 1 noted that uncertainties exist either because of lack of information – i.e., we don’t have enough information to make exact estimates about the future – or because we are involved in a

46

3 Foundations of MCDM

truly probabilistic process such as the flip of an unbiased coin. Also, Chapter 1 defined risk as “an uncertain event or condition that, if it occurs, has a positive or negative effect on one or more project objectives” [4]. Chance events then are the outcomes associated with uncertainties and these outcomes involve risk and that carry with them a positive or negative effect on one of the objectives. Uncertainties are similar to strategic decisions in that they define a set of outcomes. They differ from strategic choices in that decision makers and stakeholder groups cannot choose which outcome will occur; rather, they will experience and have to manage, for better or worse, the outcomes of such chance events. However, decisions can influence the probability and impact of certain chance events. In mathematics, uncertainties are represented by probability models. A probability model of an uncertain situation is a list of possible outcomes (chance events) accompanied by the probability of each outcome [5]. A random variable is a particular type of probability model that assigns a numerical value (i.e., effect or impact) to each outcome. Eric V. Denardo notes that the choice of the term random variable, although firmly ensconced in the literature of probability theory, is unfortunate. This is because he believes, and the authors agree, that uncertain quantity is more descriptive [6]. This makes much more sense since, simply stated, a random variable is a quantity whose value is uncertain. Random variables can be either discrete or continuous. This section focuses on discrete random variables. This is done for the purposes of introduction and simplification. An overview of continuous random variables is provided in Sections 3.5.4 through 3.5.6. In terms of mathematical notation, capital letters (e.g., X) are typically used to represent random variables and lowercase letters (e.g., x1, x2, and x3) are used to describe a value that the random variable might take [7]. Probabilities designated by the lowercase letter p1, p2, p3, etc. are used to indicate that the random variable X takes on the value x1 with a probability p1 and that it takes on the value x2 with a probability p2 and so forth [8]. A tree diagram can be used to represent a discrete random variable as presented in Figure 3.4. x1

p1

p2

X

x2 . . .

pn

xn

Figure 3.4: The random variable X as a probability tree. (From Denardo, E.V., The Science of Decision Making: A Problem Based Approach to Using Excel, John Wiley and Sons, New York, New York, 2002, 245.)

3.2 Fundamental Elements of Decision Problems

47

To help make the tree diagram example less abstract we use information contained in Table 2.1 of the case study. This table indicates that the minimum, most likely, and maximum costs to dredge transport and dispose of 325,000 cubic yards of contaminated Green River sediment is $203, $380, and $504 million, respectively. Assuming that the probabilities associated with these minimum, most likely and maximum costs are 25%, 50%, and 25%, respectively, the associated tree diagram for this random variable is provided as shown in Figure 3.5.

Minimum

25.0% $203

Sediment Dredge, Transport, Dispose Cost

Expected Value $367 Most Likely

50.0% $380

Maximum

25.0% $504

Figure 3.5: Tree diagram for Green River dredging costs.

Figure 3.5 was generated with the aid of Lumivero’s PrecisionTree software. On this diagram the probabilities associated with each outcome or chance event appear on top of the tree branches and the cost values (in millions of dollars) appear beneath the branches. In the center of the diagram, we see the number $367 (in millions) reported as the expected value. The expected value of a random variable is the probability-weighted average of the outcomes. It is also known as the mean. These terms are used interchangeably. The mean of a discrete random variable is calculated by multiplying each value that the random variable can take by the probability that this value will occur and summing the result. The expected value of a random variable X is denoted as E(X). Equation 3.2 provides the formula for calculating the expected value of a discrete random variable: Eð X Þ = x1 p1 + x2 p2 +    + xn pn

(3:2)

The expected value is a summary measure used to describe a random variable. There are two types of summary measures associated with random variables: those that measure central tendency (mean, median, and mode) and those that measure dispersion (range, variance, and standard deviation). These summary measures are described in greater detail in Sections 3.5.4 and 3.5.5. Lastly, before leaving the discussion of chance events, it should be noted that in addition to tree diagrams, discrete random variables can also be presented as a probability

48

3 Foundations of MCDM

distribution function (or probability mass function). Similar to the tree diagram, a probability distribution is a statistical function that describes all the possible values and probabilities that the random variable will take on one of the possible values. A probability distribution representing the Green River dredging costs (alternative view of Figure 3.5) is provided as shown in Figure 3.6. This figure was generated using Lumivero’s @Risk software. 60%

50%

Probability

40%

30%

Mean = 367

20%

10%

0% 150

200

250

300

350

400

450

500

550

Cost in $ Millions Figure 3.6: Probability distribution – Green River dredging cost.

3.2.4 Constraints MCDM makes use of probabilistic systems modeling techniques to find the solution that maximizes the likelihood of achieving preferred outcomes based on our values and objectives. It can be viewed as a form of constrained optimization. As defined in Chapter 1, optimization means efficiently using available resources to achieve best possible outcomes given the constraints of time, money, energy, technology, and societal preferences. Constraints represent conditions that the solution to an optimization problem must satisfy. There are many different types of constraints including: – Policy constraints – Legal and regulatory constraints – Budgetary constraints – Schedule constraints

3.2 Fundamental Elements of Decision Problems

– – – –

49

Physical constraints General constraints Integer constraints Value constraints

Most of these constraints are self-explanatory, in particular, budget and schedule constraints. Further we briefly review the other constraints in the abovementioned list. It should be noted that the above list should not be assumed to be exhaustive. There may be constraints that apply to a given decision situation and may not fit into any of these categories. Policy constraints are those that a business, or any organization, may set as guiding principles. In many instances they can be seen as decisions that have already been made. These could be policies that apply throughout an organization such as “we will not pay bribes to obtain permits or approvals in order to conduct operations in foreign countries” or they could be policies that apply to a particular project such as “we will sell this manufacturing facility because it’s no longer part of our core business.” Legal and regulatory constraints must be factored into MCDM process because reputable organizations do not knowingly break the law or fail to comply with applicable regulations. Therefore, a strategic decision cannot include any choices that are illegal or noncompliant with existing regulations governing a particular activity or endeavor. Physical constraints, in most cases, also exist within the set of known facts. Examples of these in relation to our case study are provided in Section 3.2.2. As an example of a general constraint, suppose a corporation has $500 million capital budget for the coming year and five projects in various stages of development in which they could invest some percentage of this $500 million. A general constraint would be that the percentages must sum up to 100%. Optimization software, such as Lumivero’s RiskOptimizer, allows you to specify constraints requiring decision variables to assume only integer (i.e., whole number) values. For example, if you are scheduling a fleet of delivery vehicles, a solution that calls for a fraction of a vehicle to travel a certain route would not be useful. Therefore, this optimization software, as well as other optimization software packages, allows the user to specify when only integer value solutions can be chosen. Value constraints typically refer to maximum or minimal values regarding decision variable. Referring to our fleet of delivery vehicles example, if a company only has five vehicles in its fleet, a scheduling solution that called for six vehicles would not be an acceptable solution.

50

3 Foundations of MCDM

3.2.5 Value Measures A value measure, as defined in Chapter 1, is the measuring scale used to indicate the degree of attainment of an objective, which in turn indicates how well our values are being met. As described in Section 1.5.12, value measures are criteria and are represented as the “C” in MCDM. The MCDM process involves estimating a specific numerical rating (i.e., level or score) for each alternative with respect to each identified value measure. In some cases, the value measure scores are known facts, as in “if we pursue an alternative involving a nature preserve, we will ensure the protection of 400 acres of migratory bird habitat.” In many cases, the scores for the various value measures are uncertain and therefore must be defined by random variables. Depending on the value measure and the associated alternative, discrete or continuous random variables are used to represent the value measures score.

3.2.6 Preferences In MCDM “preferences” refer to the weights that are given to the various value measures included in the decision problem. These weights are numerical values that indicate the willingness of the decision makers/stakeholders to make trade-offs. Trade-offs, as defined in Chapter 1, involve giving up a little of something valued in order to gain more of something valued even more. Trade-offs are never easy and therefore involve some rather hard thinking. Our willingness to make trade-offs is subjective and is based not only on our values but also on our emotions (discussed in Section 3.8.5). Since it is based on subjectivity, our willingness to make trade-offs can be very hard to articulate or more importantly to directly assign a number value, or weight, to each value that would adequately represent this subjectivity. Therefore, a number of techniques have been developed, and contained within the MCDM literature, to assist with measuring this subjectivity. These techniques include the analytical hierarchy procedure, swing weighting, and conjoint surveys. The use and setup of conjoint surveys to obtain criteria weights is described in Chapter 5 (Section 5.2).

3.3 Fundamental Concepts Having reviewed the fundamental elements or input parameters that are involved in an MCDM model, we are now ready to turn our attention to the fundamental concepts from the various fields of study that help establish the ways these input parameters are perceived and related. We begin with a discussion of systems engineering and systems thinking since these provide the conceptual basis for MCDM. We then move onto a discussion of probability theory, descriptive statistics, and methods for constructing decision models. We round out our review of fundamental concepts important to MCDM by

3.4 Systems Engineering and Systems Thinking

51

turning our attention to those associated with fields of finance, economics, and behavioral economics.

3.4 Systems Engineering and Systems Thinking According to the International Council of Systems Engineering (INCOSE): Systems engineering is a transdisciplinary and integrative approach to enable the successful realization, use and retirement of engineered systems using systems principles and concepts and scientific, technological and management methods. INCOSE uses the terms “engineering” and “engineered” in their widest possible sense: “the action to bring something about.” Engineered systems may be composed of any or all of people, products, services information, processes, and natural elements. The system engineering perspective is based on systems thinking. Systems thinking is a unique perspective on reality – a perspective that sharpens our awareness of wholes and how the parts within those wholes interrelate. When a system is considered as a combination of system elements, systems thinking acknowledges the primacy of the whole (system) and the primacy of the relation of the interrelationships of the system elements to the whole. Systems thinking occurs through discovery, learning, diagnosis, and dialog that lead to sensing, modeling, and talking about the real world to better understand, define, and work with systems. A systems thinker knows how systems fit into the larger context of day-to-day life, how they behave, and how to manage them [9].

3.4.1 Systems Modeling Systems engineering relies on systems modeling for purposes of understanding system properties that result or emerge from: – the parts or the elements and their individual properties, and – the relationships and interactions between and among the parts, the system, and its environment. In Applied Simulation Modeling, Andrew F. Seila, Vlatko Ceric, and Pandu Tadikamalla note that: Almost any time that a decision is made, a model is used to aid the decision maker. In many, if not most cases, the model is an implicit or ill-defined behavior model that involves relationships and scenarios such as “I believe if I make this decision, then I will get this outcome.” On the other hand, models can be overt and explicit – for example a spreadsheet model that gives mathematical relationships between decision variables (the quantities the decision maker can control) and the outcome of the decision [10].

There are three primary MCDM modeling techniques that are used to provide the mathematical relationships that exist between decision variables, uncertain events, and the outcomes of interest (i.e., those performance measures that the decision makers are

52

3 Foundations of MCDM

seeking to maximize or minimize). These include decision trees, influence diagrams, and spreadsheet simulation models. The techniques presented in this book focus primary on spreadsheet simulation models. The other two methods are discussed occasionally as a way of demonstrating concepts important to MCDM.

3.4.2 Systems Thinking MCDM relies on systems engineering and systems modeling to help the decision makers and stakeholders understand the likely outcomes of the decisions they might choose to make. As previously noted, the systems engineering perspective is based on system thinking. Therefore, this section is to provide an overview of systems thinking. This section, along with portions of Section 3.5 draws on similarly named sections contained within chapter 10 of Modern Project Management Techniques for the Environmental Remediation Industry by Timothy J. Havranek (CRC Press 1999, reprinted here with modification by permission of the publisher). What exactly is a system? If we were to look up this word in a dictionary, we find that a system is a group of elements that function together as a whole. This definition, although adequate, does nothing to inform us how to think in terms of systems. Engineers who deal in thermodynamics have a more accurate definition of a system. A thermodynamic system is defined as the matter enclosed within an arbitrary but precisely defined control volume [11]. This definition is a little more helpful in that we come to realize that a system has boundaries that separate it from its surroundings. It should be noted that this does not mean that the surroundings cannot act or impinge on the system or that the system cannot act on its surroundings. A more comprehensive definition of a system is provided by R. Buckminster Fuller in the book Synergetics. According to Fuller: A system is the first subdivision of [the] Universe into a conceivable entity separating all that is nonsimultaneously and geometrically outside of the system, ergo irrelevant, from all that is geometrically inside and irrelevant to the system; it is the remainder of [the] Universe that conceptually constitutes the system’s set of conceptually tunable and geometrically interrelatability of events [12].

Although this definition may seem quite confusing at first, it says a lot about the systems thinking required to develop a representative decision model. With this definition, Fuller states that the system is the structure itself which results from identifying the relative set of events and their relationships. In Fuller’s definition, we find discussion of the events inside of the system as well as those outside of it, similar to the thermodynamic definition of a systems. However, Fuller’s definition mentions irrelevant events, some of which can be inside of the system. For the purposes of MCDM, events can be defined as all of the previously described fundamental elements, i.e., the strategic choices, chance events, known facts, constraints, value measures, preferences, and output values of interest relevant to the problem at

3.4 Systems Engineering and Systems Thinking

53

hand. Systems thinking according to Fuller is the “conscious dismissal of irrelevancies” [13]. These irrelevancies can be placed into two categories: those too large and infrequent to influence the problem at hand (by definition, outside of the system); and those too small to play a part and so frequent as to virtually constitute the normal context in which the system operates (insignificant and inside of the system). The systems thinking Fuller describes is similar to turning a radio by dismissal of irrelevant, other frequency events [14]. To illustrate this point, Fuller would use a number of concentric circles presented as Figure 3.7. Outside of the outermost circle are those events that are too large and too infrequent to be relevant. Inside the next circle are those events which are almost relevant or as Fuller liked to say “tantalizingly relevant.” These events present a problem because one has to decide whether or not they are relevant. Anyone who has worked to develop a plan in a group setting has no doubt found that certain members of the group may think a particular issue (i.e., event) is extremely relevant, while others feel that it is not significant at all. Moving toward the center of our diagram we encounter that those events which we are certain are relevant. It is these events for which we seek interrelationships. In particular it is these events that we will use a spreadsheet model (or other decision modeling techniques such as decision trees or influence diagram) to provide the mathematical relationships between strategic decisions and their associated choices, other input parameters (i.e., known facts, chance events, constraints, value measures, and preferences), and the outcomes of interest that we are attempting to maximize or minimize. Moving further in we encounter another set of almost relevant events but this time on the micro scale. Finally, in the innermost circle, are the insignificant micro irrelevant events. Fuller suggested that each of the relevant events could be envisioned as the vertices of a polyhedral structure, with the edges of the structure representing the relationship between the events (see Figure 3.8). In our case, we will use mathematical formulas primarily within a spreadsheet environment to relate our input parameters to our outputs of interest to identify our optimum alternative. Those familiar with Fuller’s work will readily see the similarity of this figure to his most famous invention: the geodesic dome. For those not familiar with R. Buckminster Fuller, he was a philosopher, architect, inventor, mathematician, and a very early proponent of sustainability. Fuller coined the term Spaceship Earth and, in the 1960s and 1970s, was often referred to as “the planet’s friendly genius” [15]. “But equally important were a number of Fuller’s inventions that demonstrated sustainability concepts: a highly efficient three-wheeled car, a sustainable solar-powered home, and a structural building system that emphasized tension rather than compression” [16]. Fuller loved the term “synergy,” which he defined as the “behaviors of wholes, whole systems unpredicted by the behaviors of any of the system’s parts considered separately. . .” [17].

54

3 Foundations of MCDM

Macro-Irrelevancy - Too Large - Too Infrequent

Almost Relevant Lucidly Relevant

Almost Relevant Finite MicroIrrelevancy Too Small Too Frequent

Figure 3.7: Systems identification and relevant events. (From Fuller, R.B., Synergetics – Explorations in the Geometry of Thinking, Macmillan, New York, 1975, 235. Used with permission form the estate of R, Buckminster Fuller. For more information about Buckminster Fuller’s work, visit www.bfi.org.)

Figure 3.8: Polyhedral representation of system structure. (Geodesic sphere line illustration, Image ID T9E4A2, licensed from Alamy Limited 6–8 West Central, 127 Olympic Avenue, Milton Park, Abingdon, Oxon, OX14 4SA, United Kingdom)

3.5 Fundamental Concepts of Probability Theory

55

3.5 Fundamental Concepts of Probability Theory MCDM uses probabilistic modeling to account for the uncertainty of outcomes associated with the identified alternatives and their associated chance events. Therefore, MCDM draws heavily on probability theory. In fact, one of the central principles of decision analysis [and MCDM] is that uncertainty can be represented through the appropriate use of probability theory [18]. Therefore, we will review some of the basic concepts of probability theory, emphasizing those that have particular application to MCDM. In probability theory, the act of conducting a trial or taking a measurement is known as a sampling [19]. Probability theory determines the likelihood that a particular event will occur. An event (e) is one of the possible outcomes in a trail. It is important to note that an event can be numerical, discrete or continuous, dependent or independent. An example of a nonnumerical event is the toss of a coin. The roll of a pair of dice is a discrete numerical event since only certain numbers can result. The height of all male adults in the United States is an example of continuous numerical event since the heights can take on any value (within reasonable limits). Taken together, all of the possible events in a given experiment constitute a finite sampling space, which can be defined as E = (e1, e2, . . ., en). For example, given a pair of dice, the finite sample space is E = (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12). The probability of event ei is designated P(ei) and is calculated as the ratio of the total number of ways that the event can occur to the total number of outcomes in the set. Table 3.1 illustrates the possible outcomes for the roll of a pair of dice by summing the numbers associated with the possible outcomes for each die. This table can be used to calculate the probability of each possible outcome. Reading across the diagonals, from the bottom left to top right, we can count the number of ways that each event (i.e., roll outcome) can occur. For example, by counting across the center diagonal, we see that there are six ways to produce the number 7. Also, from the table we see that there are a total of 36 possible outcomes (6 × 6 = 36). Therefore, the probability of rolling the Table 3.1: Possible outcomes for a roll of a pair of dice. Outcomes of first die

Outcomes of second die

































































































56

3 Foundations of MCDM

Table 3.2: Probabilities of rolling the numbers 2 through 12. P() P() P() P() P() P() P() P() P() P() P()

= = = = = = = = = = =

/ / / / / / / / / / /

= = = = = = = = = = =

. . . . . . . . . . .

Sum

=

.

number 7 can be calculated by dividing the number 6 by 36, i.e., P(7) = 6/36 = 0.167. The probabilities of rolling the numbers 2 through 12 are presented in Table 3.2. There are two important points to note in Table 3.2. The first is that all of the probabilities are between 0 and 1, which is the first fundamental requirement of probability. This fundamental requirement is defined mathematically as follows: 0 ≤ Pðei Þ ≤ 1

(3:3)

The second important point regarding Table 3.2 is that the sum of the probabilities of rolling the numbers 2 through 12 is 1. This is a fundamental requirement of probability for a set of events which are mutually exclusive and collectively exhaustive. Mutually exclusive means that only one possible outcome can occur in a given trial. This is obviously true for the single roll of a pair of dice where the sum of the two dice can result in only one of the numbers 2 through 12. Collectively exhaustive means that there are no other possible outcomes other than those in our set. This is also true for a pair of dice (i.e., it is not possible to roll the number 13). The second fundamental rule of probability theory for mutually exclusive and collectively exhaustive events is mathematically stated as follows: n X

Pðei Þ = 1

(3:4)

i=1

Equation (3.4) is a fundamental rule of probability dealing with a combination of events; in other words, a rule of joint probability. It is an extension of the next rule, which deals with a subset n of mutually exclusive events from a finite sampling space. This rule, mathematically defined in equation (3.5), states that if two or more events are mutually exclusive, the probability that any one of the events will occur is the sum of their individual probabilities: Pðe1 or e2 or . . . ek Þ = Pðe1 Þ + Pðe2 Þ +    + Pek

(3:5)

3.5 Fundamental Concepts of Probability Theory

57

Note that the word or is an important term in understanding this equation. If we were rolling a pair of dice and wanted to know the probability of rolling a 2, 3, 4, 9, 10, 11, or 12 on the next roll, which is known as the “field bet” in the casino game of craps, the answer is found by summing up the individual probabilities for the seven “field bet” numbers as found in Table 3.2: (0.0278 + 0.0556 + 0.0833 + 0.111 + 0.0833 + 0.0556 + 0.0278) = 0.44. A player who makes the field bet wins if any of the numbers contained field bet set, i.e., 2, 3, 4, 9, 10, 11, or 12, is the result of the next roll of the dice. This same player loses if any number not contained in the field bet set is the result of the roll. Note that a player who makes such a bet has only a 44% chance of winning. To determine the probability of losing on the field bet, we could sum up the probabilities of all other numbers contained in the set of numbers that make up the field bet. However, this is not necessary. Using basic reasoning, we can conclude that if there is a 44% chance of winning, there must be 56% chance of losing. This example of winning or losing on the field bet, or any bet for that matter, provides the opportunity to bring up another fundamental law of probability. This is a law regarding complementary probabilities. Two events are said to be complementary if, when one event occurs, the other does not occur. A common designation in probability theory to indicate the complement of an event is to place a horizontal line over the letter  (not used to designate the event, for example, the complement of event A (winning) is A winning). The following equation is known as the law of complementary probabilities: Þ = 1 Pð AÞ + PðA

(3:6)

The next rule of probability, like equations (3.4) and (3.5), also deals with joint probability but instead of dependent events, this rule deals with two independent events. Typically, the events are from two different sampling spaces, but this rule can also apply to the same sampling space as long as sampling with replacement occurs such as drawing a card and returning it to a deck. When the events are from the same sampling space, this law applies to trials taken in series, as in the probability of rolling the number 3 on the first roll of a pair of dice and a 7 on the next roll of the dice. If the events are from different sampling spaces, E and G, the events can occur simultaneously. The rule of joint probability for independent events is mathematically stated as follows: Pðei and gi Þ = Pðei Þ × Pðgi Þ

(3:7)

Equation (3.7) is used extensively when chance nodes are included in decision trees or more general probability trees (also known as fault trees). We will consider the following example involving flight delays to demonstrate the use of equation (3.7). The case assumes that an individual is traveling from New York City to Los Angeles and the flight plan involves a connection in Chicago. In this example we assume that information provided by the airline’s website indicates that the flight from New York to Chicago experiences a departure delay 30% of the time. We also assume that information from this same website indicates that the flight from Chicago to Los Angeles experiences a delay

58

3 Foundations of MCDM

40% of the time. For this example, the probability of experiencing a departure delay out of New York City will be designated as P(Nd) and the probability of experiencing a delay out of Chicago as P(Cd). Let’s say our traveler is interested in determining the probability of experiencing a delay on both flights Applying equation (3.7), this probability is calculated as follows: PðNd Þ = 0.30 PðCd Þ = 0.40 PðNd and Cd Þ = 0.30 × 0.40 = 0.12 = 12% A probability tree for our airline travel example is shown in Figure 3.9. This diagram is useful for visualizing the application of equations (3.4)–(3.7). As we shall see, it is also useful for introducing other concepts and equations involving probability. Note that, on this diagram, the number 1 below the branches is used to indicate that a flight departed on time and 0 is used to indicate that a delay occurred. 40.0% 0 Departure Chicago to LA

12.0% 0

60.0% 1

18.0% 1

40.0% 0 Departure Chicago to LA

28.0% 1

60.0% 1

42.0% 2

Delay

Delay

30.0% 0

On Time

Air Travel NYC to LA

Departure NYC to Chicago Delay

On Time

70.0% 1

On Time

Figure 3.9: Example of probability tree for air travel involving a connecting flight.

A review of the diagram indicates that the probabilities shown at the endpoints of each pathway through the tree have been calculated by multiplying the probabilities moving from left to right across the tree branches. In other words, equation (3.7) has been applied to all four pathways. In addition, the endpoint probabilities of all four pathways sum to the number 1 (i.e., 100%), consistent with equation (3.4). Also, since the events at each chance node are complementary, their probabilities sum to the number 1, consistent with equation (3.4). Note that the values below the tree branches have been summed to calculate the total pathway values. A value of 0 indicates a delay occurred on both flights. A value of 1 indicates that a delay has occurred on one of the two flights. A value of 2 indicates that both flights were on time.

3.5 Fundamental Concepts of Probability Theory

59

Let’s assume that, in order to make in important meeting in Los Angeles, our traveler cannot afford a delay in both flights or on either one of the flights. To determine the probability of being late for the meeting our traveler could use one of the two methods. The first method would be to sum all the probabilities associated with pathways having endpoint values of 0 or 1 (the top three pathways). This would be an application of equation (3.5) since the four pathways represent a finite sampling space. The probabilities associated with the top three pathways are 12%, 18%, and 28% which sum to 58%. The second method would be to obtain the probability of both flights being on time (bottom pathway) which is 42% (or 0.42 as a decimal percent) and subtract this value from 100% (or 1 in terms of decimal percent). This would be an application of equation (3.6). Recall that equation (3.7) applies to independent events. Independence means that the probability of an event, such as a delay in the flight departing from Chicago to Los Angeles, is not influenced by a prior event, such as delay in the flight departing from New York City to Chicago. The probability of an event given a prior event is designated by a vertical bar such as P(B|A). This notation is read as “the probability of ‘B’ given ‘A.’” In Figure 3.9, we see that the probability of the delay in the connecting flight departing Chicago is the same (40%) regardless of what happened to the flight departing New York City. This is shown symbolically as follows: PðCd jNd Þ = PðCd Þ PðCd jNon time Þ = PðCd Þ These equations represent the application of a more general statement regarding independence in probability theory. Equation (3.8) is a mathematical statement of probability indicating independence. It simply states that the probability of event A given event B is the probability of A. Therefore, event A must be independent of B: PðAjBÞ = Pð AÞ

(3:8)

The converse of equation (3.8) may also be true, i.e., the probability of event B given event A is the probability of B. However, it is not a requirement of equation (3.8) as stated. If the converse is true the letter “A” in equation (3.8) would be replaced with the letter “B” and the letter “B” would be replaced with the letter “A” to indicate this condition of independence. In many cases, the fact that a prior event has occurred does indeed impact or “condition” the probability that a later event will occur. Experienced air travelers are often aware that a delay in an initial flight in a travel itinerary often increases the probability that there will be a delay in a later connecting flight. Conversely, experienced travelers are also aware that the fact that an initial flight departing on time often bodes well, i.e., improves the probability of the connecting flight departing on time. Note that the effects of these prior or “conditioning events” are not called out on a travel website since travelers arriving to a connecting flight can be doing so from many prior locations. Therefore, such data regarding conditioned probabilities is not available. In such

60

3 Foundations of MCDM

a case, the impact of a prior event on the probability of a later event must be based on the personal, or subjective, assessment of the traveler. Since equation (3.7) applies only to cases of independence, it is necessary to formulate a new equation that deals with conditional probability that is in the case where events are dependent. Equation (3.9) presents the equation for the joint probability of two dependent events. Pðei and gi Þ = Pðgi jei Þ × Pðei Þ

(3:9)

Continuing with our air traveler example, let’s assume that, regardless of the independent probabilities published by the airline concerning the departure of the flight from Chicago to Los Angeles, our traveler has made the subjective assessment that when the flight from NYC to Chicago is delayed, the probability of a delay in the flight from Chicago to Los Angeles has increased to 80%. In addition, our traveler has made the subjective assessment that when the flight from NYC to Chicago is on time, the probability that the flight from Chicago to Los Angeles departing on time has improved to 75%. Symbolically, these conditional probabilities can be stated as follows: PðCd jNd Þ = 0.80 PðCon time jNon time Þ = 0.75 Figure 3.10 presents a revision to Figure 3.9 based on our traveler’s assessment of the conditional probabilities regarding the departure of the flight from Chicago to LA. 80.0% 0 Departure Chicago to LA 0.2 20.0% On Time 1

24.0% 0

25.0% 0 Departure Chicago to LA 1.75 75.0% On Time 1

17.5% 1

Delay

Delay

Air Travel NYC to LA (2)

30.0% 0

6.0% 1

Departure NYC to Chicago Delay

On Time

70.0% 1

52.5% 2

Figure 3.10: Example Probability Tree for Air Travel Involving Conditional Probabilities.

Figure 3.10 indicates that given the traveler’s assessment of conditional probabilities, the probability of both flights in the itinerary being delayed has increased from 12.0% (see Figure 3.9) to 24%. In a similar fashion, the probability of both flights being on time has increased from 42.0% to 52.5%. Both of these increases are consistent with the traveler’s intuition regarding the conditioning of the prior flight’s status on the

3.5 Fundamental Concepts of Probability Theory

61

departure of the connecting flight’s status. A review of Figures 3.9 and 3.10 indicates that joint probabilities associated with the endpoint involving a delay of just one of the flights have decreased. This is consistent with the traveler’s intuition and with equation (3.4) since the probabilities of a finite set of events must sum to 1.

3.5.1 Bayes’ Formula and Subjective Probabilities When introducing fundamental formulas and concepts of probability theory it is common to use examples involving rolling dice, tossing coins, drawing cards, or the results of repeatable events such as plane departure information whereby information can be stored in a database for later analysis. The probabilities for such events are based on the notion of relative frequency, i.e., the number of times the event under consideration occurs in relation to the total population of events. This relative frequency can be determined analytically based on the number of ways an event can occur relative to the whole population of events, as presented in Table 3.1 regarding the roll of a set of dice. It can also be done experimentally based on actual data such as the number of late departures for a given flight number relative to the total number of times that same flight has departed over the past 10 years. Classical probability theory regards probability as an immutable number based on relative frequency. All the examples and formulas presented up to the point that conditional probabilities were introduced are based on a classical probability theory. To demonstrate conditional probabilities, we introduced the example whereby the probability of a connecting flight’s delayed or on-time departure is based on our traveler’s subjective assessment. This concept of probabilities being based on subjective assessment, rather than the result of repeatable trials, makes many who side with classical probability theory uncomfortable. Such individuals who hold to classical probability theory are known as “frequentists.” However, in nearly every facet of everyday life, individuals often speak of probabilities as measures of our degree of subjective belief based on the weight of prior evidence. For example, an individual sitting on a jury in a court of law might have a subjective belief (or assessment) that, based on the evidence presented thus far in the trial, there is an 80% probability that the defendant is guilty. As more evidence is brought to light, the juror’s assessment of probability of guilt may go up or down. In other words, the assessment of probability is conditioned by, and based on, our prior knowledge or weight of evidence. Thomas Bayes was an eighteenth-century mathematician famous for Bayes’ Formula which describes the probability of an event based on prior knowledge related to the event. To introduce the formula and help make it more understandable we make use of a simplified form of equation (3.9), presented as follows: PðA and BÞ = PðBjAÞ × Pð AÞ

(3:10)

62

3 Foundations of MCDM

In this simplified form, we have removed the events that included subscripts to help make the formula a little less messy. The letters A and B in the above formula indicate events that have occurred; in this case, we will use the letter A to indicate a delay in the departure of the flight out of New York (previously designated at Nd) and B to indicate a delay in the departure of the flight out of Chicago (previously Cd). Therefore, P(A and B) refers to the joint probability for the occurrence of events A and B. These joint probabilities are the endpoints of the branches of the probability trees in our previous examples. It should be noted that for reasons of symmetry, the simplified version of equation (3.9) can also be restated as PðA and BÞ = PðAjBÞ × PðBÞ

(3:11)

If we consider the case pathway in Figure 3.10 where both flights are delayed (top branches), we find PðA and BÞ = 0.24 Looking again at the simplified versions of equation (3.9) (i.e., equations (3.10) and (3.11)) we see that they can be rearranged as follows: PðBjAÞ =

PðA and BÞ Pð AÞ

(3:12)

PðAjBÞ =

PðA and BÞ PðBÞ

(3:13)

To assist with understanding equation (3.12), we refer again to Figure 3.10. Based on the information provided in this figure we see that PðBjAÞ =

PðA and BÞ 0.24 = = 0.80 Pð A Þ 0.30

The result 0.80 is as expected for the P(B|A). The various equations related to conditioned probabilities presented so far have been leading up to a final and most profound application of Bayes’ Formula: the calculation of a posteriori (“from the latter”) probabilities. We noted from our traveler example that our air traveler was able to subjectively estimate the probability of the status of their connecting flight out of Chicago (delay or not delay) based on knowledge that the status of their originating flight out of New York City (delay or not delay). In other words, our traveler is able to estimate the status of the connecting flight after having a priori (“from the earlier”) knowledge of the originating flight. Symbolically the traveler is able to subjectively estimate PðBjAÞ A Þ and PðBj However, the traveler would likely find it very difficult to do the reverse; that is, to estimate the status of the originating flight given the status of the connecting flight,

3.5 Fundamental Concepts of Probability Theory

63

that is, to estimate a posteriori P(A|B). This can be done using Bayes’ Formula, presented here as equation (3.14) PðAjBÞ =

PðBjAÞ × Pð AÞ  Þ × PðA Þ PðBjAÞ × Pð AÞ + PðBjA

(3:14)

Equation (3.14) can be used to perform what is known as a Bayesian reversal. At first look, Bayes’ Formula seems quite confusing. However, further inspection indicates that the equation is derived by applying algebraic substitution to the right-hand side of equation (3.13), i.e., the equation used for calculating PðAjBÞ. We begin by working to replace the numerator of equation (3.13), i.e., PðA and BÞ, using an expression that contains variables that are known. A review of equations (3.10) and (3.11) indicates that PðA and BÞ ¼ PðBjAÞPð AÞ ¼ PðAjBÞPðBÞ Therefore, we can replace PðA and BÞ with either of the expressions following equal signs. The second expression is not helpful since it contains PðAjBÞ, which is what we are seeking to determine. The first expression contains variables which a traveler already knows: PðBjAÞ and Pð AÞ. Recall that PðBjAÞ represents the probability of delay in the departure from Chicago to Los Angeles given the delay in the flight departing from New York City to Chicago. Also recall that Pð AÞ represents the probability of delay in the flight departing from New York City to Chicago. These two probabilities are 80% and 30% and are readily seen in Figure 3.10. Replacing the denominator of equation (3.13) means that we need to find an expression for PðBÞ, the probability of the flight departing from Chicago to Los Angeles. This is where Bayes’ Formula can be confusing because it seems we already know PðBÞ, i.e., the independent probability of the delay departure of the flight from Chicago to New York City, which was previously given as 40%. However, we are seeking to know PðBÞ, when although, apparently independent, it is derived from our knowledge that event B is dependent on event A. To calculate PðBÞ, we can use the rule of joint probability (equation 3.5) and the law of complementary probabilities to develop the following equation:  ÞPðA Þ PðBÞ ¼ PðBjAÞPð AÞ þ PðB=A

(3:15)

A review of Figure 3.10 indicates that this equation simply sums the two pathways where event B has occurred, i.e., the flight departing from Chicago to Los Angeles has been delayed. This probability is 41.5% (i.e., 24.0% + 17.5% = 41.5%). To continue with our air traveler example one last time, let’s say the night before the traveler’s flight plan there are news reports of thunderstorms in the Chicago area which will delay departing flights. Our traveler is able to estimate P(B|A) but is not confident in estimating P(A|B), i.e., the probability that the flight leaving New York will be delayed given a delay in the flight departing Chicago. However, this traveler

64

3 Foundations of MCDM

has all the information needed to apply (equation (3.14)). Using the traveler’s previous conditional probabilities and the weather forecast, the traveler knows the following: PðBjAÞ = 0.80 A  Þ = 0.75 PðBj Pð AÞ = 0.30 By the rule of complementarity, the traveler also knows that  Þ = 0.25 PðBjA  Þ = 0.70 PðA Applying equation (3.14), we have: PðAjBÞ =

0.80 × 0.30 = 0.578 0.80 × 0.30 + 0.25 × 0.70

Based on updated information about the regarding the possible delay of the connecting flight, the traveler now knows that the probability of the originating flight being delayed is approximately 58%. Therefore given new information of flight delays out of Chicago are traveler’s estimate of the flight being delayed out of New Your City to Chicago has nearly doubled going from 30% to 58%. Upon seeing this result, one might say well that this seems to work out mathematically but why would weather in Chicago affect the departure of flight from New York. Recall our flight from New York is going to Chicago. If planes are not able to leave Chicago due to weather, it also means that planes to Chicago may not be able to land due to weather; therefore, the plane to Chicago may be delayed in New York until it has been determined that this plane will be able to land upon arrival to Chicago. Before leaving the discussion of Bayes’ Formula, it should be noted that individuals and mathematicians who accept that probabilities are not immutable and can be updated based on new information are known as “Bayesians” as opposed to “Frequentists.” However, one does not need to side with one group or the other. In those situations where repeatable experiments are possible, one can apply classical probability theory and in those instances side with the Frequentists. On the other hand, when repeatable experiments are not possible, one can make use of conditional probabilities and in that instance side with the Bayesians. Bayes’ Formula is an important rule of probability theory and is widely used by those working in the areas of data mining and predictive analytics. According to Paul Newendorp and John Schuyler, “The Google Empire is built around Bayesian Analysis” [20]. Some of the more important uses in MCDM and decision analysis in general include value-of-information calculations and Bayesian reversals. Given these important applications, no summary of fundamental concepts of probability theory would be complete without a discussion of this formula.

3.5 Fundamental Concepts of Probability Theory

65

3.5.2 Describing Experimental Data Having reviewed some of the basic laws of probability theory as well as a more advanced concept of Bayes’ Formula, we will now turn our attention to the use of probability theory in describing experimental data. Describing experimental data involves the use of graphing techniques as well as calculated descriptive statistics which measure the central tendency of data as well as their degree of dispersion. Familiarity with common techniques for describing experimental data is important not only to understand the results of models used for the purposes of MCDM but also, and perhaps even more importantly, to properly select input probability distribution functions used to represent model input parameters involving uncertainty.

3.5.3 Graphical Representations of Experimental Data Common graphical representations for describing experimental data include the frequency histogram, the probability density function, and cumulative distribution function. 3.5.3.1 Frequency Histograms A frequency histogram for a set of experimental data is developed by first dividing the total range of the data set (as determined by the minimum and maximum values in the set) into equal width intervals and then counting the number of occurrences within each interval. Once this is done, the histogram is created by drawing vertical bars with height proportional to the number or frequency of occurrences within each interval. Figure 3.11 presents a histogram used to represent the outcomes associated with rolling a pair of dice 10,000 times. The outcomes of this dice rolling experiment were produced using @Risk to perform a 10,000 iteration Monte Carlo simulation. Note that the relative frequencies (or probabilities in decimal percent) recorded at the top of each histogram bars equal the calculated or theoretical relative frequencies presented in Table 3.2. The equivalence between theoretical expectation and experimental data is a result of the experiment being repeatedly performed so many times (i.e., 10,000 times) and is a demonstration of the strong law of large numbers. It is the principle upon which Monte Carlo simulation is built [21]. The strong law of large numbers basically states that the larger the sample size (i.e., the greater the number of iterations) the closer the output distribution will be to the theoretical distribution [22]. Note that the histogram presented in Figure 3.11 could have been developed using the information contained in Table 3.2. Such a histogram would be known as the theoretical probability distribution for the roll of a pair of dice. Figure 3.12 is a histogram produced using cost data pertaining to what is known as a Phase B investigation (or extended investigation) in the environmental remediation industry. Such investigations are performed to determine the extent of soil and

66

3 Foundations of MCDM

0.18

0.1667

0.16 0.1389

Relative Frequency

0.14 0.12

0.1389

0.1111

0.1111

0.10 0.0833

0.0833

0.08 0.0556

0.06 0.04

0.0556

0.0287

0.0278

0.02 0.00 2

3

4

5

6

7

8

9

10

11

12

Figure 3.11: Frequency histogram for an experiment involving 10,000 rolls of a pair of dice.

groundwater impacted by a contaminant of concern at a given site once a preliminary investigation, i.e., Phase A, indicates that contamination is present or likely present. This histogram in Figure 3.12 is from a portfolio of gasoline service station sites. 0.14

0.12

Relative Frequency

0.1

0.08

0.06

0.04

0.02

Phase B Investigation Costs Figure 3.12: Histogram of Phase B investigation cost data.

$4,53,805

$4,31,212

$4,08,620

$3,86,027

$3,63,435

$3,40,842

$3,18,250

$2,95,657

$2,73,064

$2,50,472

$2,27,879

$2,05,287

$1,82,694

$1,60,102

$1,37,509

$1,14,916

$92,324

$69,731

$47,139

$24,546

0

3.5 Fundamental Concepts of Probability Theory

67

3.5.3.2 Discrete and Continuous Distributions The histograms shown in Figures 3.11 and 3.12 are both graphical representations of probability distributions in that they depict the probability that an experiment will produce a particular result, as in the case of Figure 3.11 (the set of integers 2 through 12); or as range of values, as in the case of Figure 3.12, where the width of each bar (i.e., interval or bin) represents $22,592. The dollar value below the histogram bars of Figure 3.12 is the interval midpoint. Figures 3.11 and 3.12 are actually representative of the most distinguishing property of probability distributions, which is whether they are discrete or continuous [23]. The concept of a discrete probability distribution function to represent a random variable was previously introduced in Section 3.2.3 and with Figure 3.6. The formal definition of discrete distribution is distribution that may take on one of a set of identifiable values, each having a calculatable probability of occurrence [24]. The distribution associated with the possible outcomes for a pair of dice and presented in Figure 3.11 fulfills the definition of a discrete distribution. Figure 3.6 is another example of a discrete distribution. Other examples of variables that can only take on discrete distributions include the number of people that could arrive at an airport transportation security check station in a given time frame, the number of successful oil and gas wells associated with a 10-well exploratory program, the number of children born in a large city hospital on New Year’s Day. In all these examples, the variables must take on specific whole unit number values. Section 3.5.6 provides a description of commonly used discrete probability distributions. Although the appearance of the histogram in Figure 3.12 gives the impression that this distribution is discrete, it is actually a continuous distribution. This is because the bars on the graph represent a range of values. By definition, a continuous distribution is one that is used to represent a variable that can take on any value within a defined range (i.e., the domain of the distribution) [25]. The fact that the variable can take on any value makes it impossible to calculate the probability of a particular number within the range of the entire distribution or within a particular interval. However, integral calculus can be used to calculate the probability of an increasingly smaller bin around the value of interest. Lastly regarding Figure 3.12, one can imagine that if additional data could be collected, perhaps from thousands of sites, the number of bins could be increased substantially and if this were done, this distribution would begin to take on a rather smooth appearance, i.e., thus looking more representative of a continuous distribution. The authors have developed risk models to estimate the total cost liabilities associated with the portfolios of gasoline service stations requiring environmental investigation and cleanup to address soil and groundwater impacts. Each site in these portfolios would be at different phases in the cleanup process. Data such as that represented in Figure 3.12 were used to fit continuous probability distributions representative of each project phase.

68

3 Foundations of MCDM

Figure 3.13 presents a theoretical probability distribution that was fit to the data underlying Figure 3.12. This “fitting” was done using the distribution fitting feature of Lumivero’s @Risk program.

Figure 3.13: Fitted distribution Phase B investigation costs.

The fitted distribution presented in Figure 3.13 is a particular type of distribution known as the Weibull distribution. Section 3.5.6 provides a discussion of additional distinguishing properties of probability distributions, i.e., beyond discrete and continuous. For now, we note that the Weibull distribution is a continuous distribution that is constrained on the lower end to values greater than or equal to 0. This is a useful feature since it would not make sense for this model to sample values less than 0 (i.e., negative costs). The upper end of the Weibull distribution extends to infinity, albeit at infinitesimally small probability. In general, this is not a problem since excessively high values are not likely sampled. For example, the 99th percentile of the distribution shown in Figure 3.13 is $433,100. This means that 99% of the values sampled from this distribution while running a model will be less than or equal to $433,100. Conversely, there is only a 1% chance that the sampled values will exceed $433,110. However, to prevent excessively high values, the truncate feature @Risk can be used when defining a distribution to ensure that it is not sampled above a reasonable maximum, such as $1 million. Figure 3.14 is a screenshot showing the application of this setting applied to the Weibull distribution.

3.5 Fundamental Concepts of Probability Theory

69

Figure 3.14: Application of @Risk’s Define Distribution Truncate Setting.

It should be noted that in common usage, the term probability distribution function is applied to both discrete and continuous distributions. However, formally, the term probability distribution function applies only to discrete distributions. The proper term for a continuous distribution is probability density function. This term is used because the probability of any particular value of X within the range of the distribution is extremely small due to the fact that the distribution allocates probabilities, which must sum to 1, among an infinite number of values of X. Note, that the scale of the vertical axis in Figure 3.13 is in units of 10–6 or millionths. This indicates that the probability of a particular value (in reality for very same range around the value of interest) is very small. However, what is most important about a probability density function shown in Figure 3.13 (and all continuous distributions) is its shape, which indicates the relative probability of values of X within the range of the distribution. 3.5.3.3 Cumulative Distribution Functions – Discrete and Continuous Distributions The cumulative distribution function, F(x), gives the probability of a numerical event being less than or equal to a chosen value of x within the domain of the distribution. Cumulative distribution functions exist for both discrete and continuous distributions. The cumulative distribution curve is developed by summing the area of the underlying probability distribution from the value furthest left to the value of interest, which is the same as integrating the underlying probability distribution/density function. In other words, if f(x) represents the probability distribution function or probability density function, the cumulative distribution function can be expressed mathematically as follows:

70

3 Foundations of MCDM

ð F ð x Þ = Pð X ≤ x Þ =

f ðxÞdx

(3:16)

Figure 3.15 presents the cumulative distribution function that corresponds to the probability distribution function representing possible outcomes for a pair of dice (i.e., Figure 3.11). Note that this figure has stair-step shape which is characteristic of all discrete distributions. The number of steps is equivalent to the number of values that make up the distribution. Note that it is possible to perform a manual integration (i.e., obtain the area under the curve) of a discrete distribution by simply summing the probabilities from the furthest value left to the value of interest. For example, to determine the probability of rolling a value less than or equal to the number nine, we can simply sum the probabilities for the each of the numbers two through nine (see Table 3.2) which is 0.8333% or 83.33%. This is consistent with the curve presented in Figure 3.15. 100% 90%

Cummulative Probability

80% 70% 60% 50% 40% 30% 20% 10% 0% 2

3

4

5

6

7

8

9

10

11

12

Figure 3.15: Cumulative distribution function for the outcomes of a pair of dice.

Figure 3.16 presents the cumulative distribution function that corresponds to the Weibull probability density function introduced in Figure 3.13. Note that this curve is smooth and upward trending and has a somewhat squashed or leaning “S”-shaped appearance. All cumulative distribution functions are upward trending and those representing continuous distributions tend to be “S” shaped in appearance.

3.5.4 Measures of Central Tendency The central tendency or “location of the data” refers to a value which is typical of all sample observations. Three common measures of the central tendency of the data are the mean, median, and the mode. These three measurements are included in Figures 3.13 and

3.5 Fundamental Concepts of Probability Theory

71

100% 90%

70% 60% 50% 40%

0% 0

50,000

1,00,000

+1 SD = 237,697

10%

Mean = 144,517

-1 SD = 51,338

20%

Median = 125,201

30% Mode = 76,092

Cumulative Probabilty

80%

1,50,000

2,00,000

2,50,000

3,00,000

3,50,000

4,00,000

4,50,000

Phase B Investigation Costs

Figure 3.16: Cumulative distribution function for Phase B investigation costs.

3.16. The mean is what most people call the “average.” It is the sum of all measurements divided by the number of measurements. A weighted average is actually a weighted mean. The expected value, first introduced in Section 3.2.3, is a probability weighted average. The terms expected value and mean can be used interchangeably. Equation (3.2) provided the formula for the expected value or mean of a discrete distribution. The general formula for the mean of a continuous distribution is provided as in equation (3.18). This equation uses lowercase Greek letter mu (μ) to symbolize the mean, as is commonly used in probability theory: +ð∞

xf ðxÞdx

μ=

(3:17)

−∞

The mode is the value that occurs more frequently than other values associated with a discrete or continuous distribution. It is where the concentration of the data is the greatest. The mode can be read directly from a frequency histogram or probability density function by looking for the highest peak in the curve. It is a relatively quick measure of central tendency. The median is the point on a frequency histogram or probability density function that partitions the total set of measurement into two sets of equal numbers. The median is the middle point of all observations. Percentile rank is related to the concept of the median. The median could also be called the 50th percentile rank. In a similar fashion, the 90th percentile rank would be that point on the horizontal axis of the cumulative distribution function that corresponds to 90% cumulative probability, i.e., F(x) = 90%. Another important distinguishing characteristic of probability distribution is whether they are symmetrical or asymmetrical. The values of the mean, median, and mode are

72

3 Foundations of MCDM

the same for all symmetrical distributions. Figure 3.11 (dice histogram) is an example of a symmetrical discrete distribution. The mean, median, and mode for this distribution are all the same, i.e., the number seven. The Weibull distribution presented in Figure 3.13 is an example of an asymmetrical continuous distribution. The mean, median, and mode for this distribution are 144,517, 125,201, and 76,092, respectively. The continuous distribution that most people are familiar with is the normal (Gaussian) distribution which is sometimes referred as the “bell curve.” The normal distribution is a symmetrical continuous distribution. The log-normal distribution, which is familiar to many individuals working in science, engineering, and economics, like the Weibull distribution, is an example of an asymmetrical continuous distribution.

3.5.5 Measures of Dispersion Although the mean, median, and mode provide information about the central tendency or “location of the data,” they indicate very little about the way in which the data is spread out or dispersed. Figure 3.17 illustrates why having an idea of dispersion of data is important. This figure shows example cost probability density functions for two competing strategies that could be employed for completing a technical project. Both strategies have the same mean even though the shape of their cost probability density function is significantly different. 16%

12% 10% 8% 6% 4% Mean = 500

Relative Frequency

14%

2% 0% 100

200

300

400

500

Cost in $ Millions

600

700 Strategy A

800 Strategy B

900

Figure 3.17: Cost probability distributions for competing project strategies.

Strategy B has a much wider distribution of cost data. Because of this, one could say that this strategy is much riskier than Strategy A. Let’s say that the available project budget is $550 million. Assuming other factors are equal, such as the revenue associated with each strategy, then Strategy A should be chosen for implementation. This is because there is much better chance of not exceeding the established budget with this strategy.

3.5 Fundamental Concepts of Probability Theory

73

The three common measures of dispersion are the range, variance, and standard deviation. These parameters are indicators of the dispersion of the data around a central location, which for practical purposes is the sample mean. The range is the simplest measure of dispersion and is found simply by subtracting the largest sample value from the smallest. The range can be greatly affected by extreme values (i.e., low probability values) and is therefore limited in practical use. In addition, the range for unbounded distributions, such as the normal distribution, extends from negative infinity to positive infinity, making it impractical as a measure of dispersion. Variance, also known as the mean squared deviation, is more useful than the range because it considers all values from a sample set. Equation (3.18) provides the formula for sample variance (and for a discrete distribution) and equation (3.19) provides the formula for a general continuous distribution. In these equations the lowercase Greek letter sigma (σ) is used to symbolize standard deviation. Since variance is the squared standard deviation, it is symbolized by σ 2 : Pn σ2 =

ðxi − μÞ2 n−1

i=n

(3:18)

+ð∞

ðx − μÞ2 f ðxÞdx

σ2 =

(3:19)

−∞

The larger the variance, the greater the degree of dispersion. Therefore, it is useful when comparing two sample sets or probability distributions (discrete or continuous). In Figure 3.17, Strategy B would have a much greater variance than Strategy A. Variance is measured in square units. Therefore, if the sample data being analyzed is in dollars, the variance would be in dollars squared, which is somewhat nonsensical. Note that equation for variance involves subtracting the mean from the various values of all values of x in the data set. If these differences were not squared prior to summing, the summation would always be zero. Therefore, squaring the difference is necessary, but results in the issue of squared units. This problem is addressed by taking the square root of the variance which, by definition, is the standard deviation.

3.5.6 Distinguishing Properties of Probability Distributions Three distinguishing properties of probability distributions help modelers ensure that the distributions properly represent the uncertain input parameter in the manner intended: – Discrete or continuous – Bounded or unbounded – Parametric or nonparametric The difference between discrete and continuous properties has been described at length earlier in this chapter, with particular attention provided in Section 3.5.3.3.

74

3 Foundations of MCDM

A bounded distribution is one that is confined to line between two values. An example of a bounded distribution is the uniform distribution where the values lie between a minimum and maximum values. An unbounded distribution can theoretically extend from minus infinity to plus infinity. The normal distribution is an example of unbounded distribution. A distribution that is constrained at one end or the other is said to be partially constrained. The Weibull distribution as presented in Figure 3.13 is an example of a partially constrained distribution. In Risk Analysis – A Quantitative Guide, David Vose points out that there is a very useful distinction to be made between model-based parametric and empirical, nonparametric distributions [26]. According to Vose, a model-based distribution is one whose shape is born of the mathematics of describing a theoretical problem, while an empirical distribution is one whose mathematics is described by the shape that is required. By way of example, Vose shows how both exponential and lognormal distributions are model-based: the exponential distribution is the direct result of assuming that the rate of decay of x is proportional to x and that a lognormal distribution is derived from assuming that ln(x) is normally distributed [26]. An example of an empirical distribution is the Triang distribution which is defined by its minimum, mode, and maximum. There is a wide variety of probability distributions that can be used to represent inputs to probabilistic systems models for MCDM. A detailed presentation of the various distributions is beyond the scope of this book. However, in the following section we provide an overview of those distributions most useful for MCDM. For a comprehensive coverage of statistical distributions, including their mathematics and application readers are referred to Statistical Analysis, Second Edition by Merran Evans, Nicholas Hastings, and Brian Peacock (1993 John Wiley and Sons). In addition, Chapter 6 of Risk Analysis, Quantitative Guide, Second Edition by David Vose (2000 John Wiley and Sons) provides an in-depth review of a wide variety of probability distributions. Last, the @Risk software includes a library of 107 probability distributions that can be used for modeling purposes. When adding a distribution to a model using the @Risk Define Distribution button, the user can access details about each of the available distributions regarding their characteristics (discrete, continuous, bounded, unbounded, etc.) and the areas where they are commonly used. Figure 3.18 provides a screenshot of this feature.

The @Risk Resources button can be used to obtain access to the online reference manual and comprehensive information regarding each available probability distribution function including syntax, use guidelines, parameters, domain, and formulas for the density and cumulative distribution functions.

3.5 Fundamental Concepts of Probability Theory

75

Figure 3.18: Screenshot of @Risk Define Distribution Feature.

3.5.7 Probability Distributions Most Useful for MCDM Table 3.3 provides a summary to the distributions that we have found most useful for MCDM. Table 3.3: Probability distributions most useful for MCDM. Distribution Type name

@Risk Syntax

MCDM use

Bernoulli

Discrete, bounded, parametric

RiskBernoulli(p)

Used to model events such as regulatory approvals, property sale, environmental cleanup goals achieved, and lawsuits.

Binomial

Discrete, bounded, parametric

RiskBinomial(n,p)

Can be used in place of Bernoulli distribution when n is set to .

Compound

Continuous, unbounded, parametric

RiskCompound (dist#,dist#)

Used to model low probability high impact events. However, it is often better to keep the two distributions separate for purposes of later sensitivity analysis.

76

3 Foundations of MCDM

Table 3.3 (continued) Distribution Type name

@Risk Syntax

MCDM use

Cumul

Continuous, bounded, empirical

RiskCumul (min,max,{x,x, x. . .},{cp, cp, cp. . .})

Can be used to represent a distribution based on information elicited from subject-matter expert.

Discrete

Discrete, bounded, empirical

RiskDiscrete ({X,X, . . ., Xn}, {p,p, . . ., pn})

Used when data available fits into this format. Can be used to represent a distribution based on information elicited from subject-matter expert.

Intuniform

Discrete, bounded, empirical

RiskIntUniform (minimum, maximum)

Used to model the number of days to receive a permit approval or complete a task.

General

Continuous, bounded, empirical

RiskGeneral(min,max, {X,X, . . ., Xn},{p,p, . . .,pn})

Sometimes used to represent a distribution based on information elicited from a subjectmatter expert.

Lognormal

Continuous, left bounded, parametric

RiskLognorm(mean, standard deviation)

Usually used when it is fit to an available data set.

Normal

Continuous, unbounded, parametric

RiskNormal(mean, standard deviation)

Usually used when it is fit to an available data set.

PERT

Continuous, bounded, parametric

RiskPert (min, m.likely, max)

Preferred distribution for shaping a distribution based on information elicited from a subjectmatter expert. Can look normal, lognormal, and skewed left or right based on the minimum, most likely, and maximum values provided.

Triang

Continuous, bounded, empirical

RiskTriang(minimum, m.likely,maximum)

Similar to PERT but avoided by the authors. This distribution has an odd shape typically not found in data sets associated with phenomena in nature or economics. Also, the mean is overly influenced by the extreme values.

Uniform

Continuous, bounded, empirical

RiskUniform (minimum,maximum)

Used when there is little information about the parameter in question. This distribution is sometimes called the “no knowledge” distribution, as in, “we have no idea what the uncertain value will be, except that we believe it well be somewhere between  and  (for example).”

3.6 Fundamental Concepts of Finance

77

3.6 Fundamental Concepts of Finance Like the concepts from other fields of study, MCDM draws on only the most basic concepts of finance, i.e., the time value of money and NPV. In addition, as we explain in this section, cash flow models developed for analyzing competing alternatives are more useful when they are structured using only the most basic time value of money formulas in a stepwise process.

3.6.1 Time Value of Money The time value of money is perhaps the most basic of financial principles. Simply stated, this principle says that a dollar today is worth more than a dollar tomorrow. There are two reasons that this is true. The first is the opportunity cost and the second is inflation. Opportunity cost is the benefit that one foregoes when choosing one alternative over another. Therefore, having money today, rather than say one year from now provides individuals (or business, governments, or other entities) with a number of opportunities such as buying something that is needed (or wanted), paying off debts, or investing in other assets. Therefore, individuals, businesses, and other entities seek compensation in the form of interest whenever lending or investing money (e.g., in bonds and stocks) and foregoing the opportunity to use it today. Similarly, the opportunity cost of spending money today is the foregone interest of waiting a year. The second reason that a dollar today is worth more than tomorrow is inflation. Inflation is the phenomenon where prices of goods and services increase over time. Therefore, as a result of inflation, the same amount of money will purchase less goods and services in the future than it would today. Therefore, the interest rate for lending money will include the expected rate of inflation. The combination of opportunity cost and inflation leads directly to the understanding that it is better to receive payments due as soon as possible and delay payments owed as late as possible.

3.6.2 Net Present Value Before introducing the concept of NPV, it is useful to begin by introducing other more basic formulas related to the time value of money, i.e., the single payment present value (PV) formula and the single payment compound amount formula. These two formulas are provided below in equations (3.20) and (3.21), respectively. In these equations, P represents the present amount and Fn represents a future amount received in time period n. The letter i in both equations represents the effective interest rate per period (in decimal percent) and the small letter n represents the number of periods typically in years for an MCDM. It is important to consider whether i is a “real”

78

3 Foundations of MCDM

interest rate, which means it excludes inflation, or a nominal rate, which reflects the “real” interest rate plus the expected rate of inflation. Note, when using a nominal rate, it is important to make sure that Fn also reflects the expected inflation rate. Doing so ensures that inflation will not affect P: P=

Fn ð1 + iÞn

Fn = Pð1 + iÞn

(3:20) (3:21)

To demonstrate the use of equation (3.21), the single payment PV formula, suppose that a company must make a payment of $50,000 five years into the future, i.e., from the present time. This company uses 8% as its discount rate (the value i in the equation). Using the information, the PV of the payment of $50,000 five years in the future is approximately $34,000 (see below). In other words, for this company, a payment of $50,000 five years in the future is equivalent to a payment of $34,000 today: P=

$50,000 ð1 + 0.08Þ5

= $34,029

It should be noted that the discount rate varies by company and it is based on the company’s weighted average cost of capital (WACC) which is defined as the average rate of return demanded by debt and equity investors. In other words, the average interest rate that the company must pay to borrow money. A deeper discussion of WACC is beyond the scope of this book. However, the WACC for most publicly traded companies can often be found via an internet search. An important distinction regarding WACC is that it represents a nominal rate meaning that it includes the expected rate of inflation. To be certain that the proper discount rate is being used, those building the model should confirm with the corporation’s finance department. Looking again at our company’s required payment of $50,000 in 5 years, the application of equation (3.21) works fine if the payment is a known fixed amount such as a bond payment. However, what if this payment is for something like replacing a piece of equipment that is expected to wear out in five years and this piece of equipment costs $50,000 in present dollars. This means that before applying equation (3.21) we would first need to convert our present dollars to future dollars based on our expected rate of inflation and then bring it back to present dollars at the company’s WACC. This two-step process is as follows: F = $50,000ð1 + 0.03Þ5 = $57,965 P=

$57,965 ð1 + 0.08Þ5

= $39,499

Notice that once we account for inflation, the company’s PV for this expenditure is approximately $39,500, nearly $5,500 more than the PV cost when not accounting for

3.6 Fundamental Concepts of Finance

79

inflation. The difference of $5,500 dollars when accounting for inflation brings up the importance of including the effect of inflation when performing PV calculations. Imagine if this payment were an operating expense (OpEx) that will be made every year for as long as the facility is in operation. Excluding the effect of inflation on these expenditures would greatly underestimate their PV cost. 3.6.2.1 Nominal Versus Real Dollars Whenever performing cash flow computations, it is important to understand the type of cash flows you are working with and match the proper inflation and discount rates to your cash flows. Economists are careful to note the difference between nominal and real dollars. Nominal dollars refer to the dollars that will be required at the time that the expenditure takes place (sometimes referred to as current dollars). Nominal dollars, therefore, are inflated dollars. Real dollars are based on the real interest rate which removes the effect of inflation. In essence, real dollars refer to the amount of purchasing power of the dollars which decreases as inflation increases. The most important concept to bear in mind when performing cash flow calculations is to match the inflation and discounting rates to the type of cash flow you are working with. This means that: – If your cash flows are in nominal (inflated) dollars use a nominal discount rate. – If your cash flows are in real (uninflated dollars) use a real discount rate. The abovementioned two-step process that resulted in a PV of $39,499 is the result of matching nominal dollars (inflated) with the nominal interest rate. This is because the nominal interest rate is the rate at which one is able to borrow money. For a corporation, its WACC is its nominal interest rate. In general, we recommend using nominal dollars and nominal interest rates when performing cash flow computation analysis for purposes of MCDM. The cash flow model included in the MCDM template provided along with this book assumes the use of nominal dollars and nominal interest rates. However, the use of real (uninflated dollars) and a real discount rate will result in the exact same values as when using nominal dollars and the nominal discount rate. This can be done within the cash flow modeling included in the MCDM template by setting the inflation rate to zero and using a real discount rate instead of the nominal rate. The following sections provide the equation for calculating the real interest rate. The most important thing is to remain consistent in applying nominal discount rates to nominal dollars and real discount rates to real dollars. One should be careful to never mix values using real (uninflated) dollars and a nominal interest. The reader may note that this was done in the very first calculation in this section, i.e., the one that resulted in a cost of $34,029. However, this was done for demonstration purposes only and to later show the effect of inflation. Lastly, once a decision has been made

80

3 Foundations of MCDM

regarding modeling in nominal or real dollars, this decision should remain in place throughout the modeling process. 3.6.2.2 Real Discount Rate This section is provided for those who would like a better understanding of the real interest rate. The real discount rate is a rate that has been adjusted to remove the effects of inflation. It can be calculated using equation (3.22) (which is preferred) or approximated using equation (3.23) (for purposes of quick analysis):   1 + nominal discount rate −1 (3:22) Real discount rate = 1 + inflation rate Real discount rate ≈ nominal discount rate − inflation rate

(3:23)

Applying equation (3.22) to our previous example involving 8% nominal discount rate and a 3% inflation rate results in real discount rate of 4.85% (see further):   1 + 0.08 − 1 = 0.0485 Real discount rate = 1 + 0.03 The real interest rate can be applied to the uninflated (real) payment of $50,000 expected in 5 years into the future using equation (3.20), the single payment PV formula, to arrive at a result of $39,499 which is the same when the nominal discount rate was applied to nominal dollars, as expected (see further): PV =

$50,000 ð1 + 0.0485Þ5

= $39,499

3.6.2.3 Advantages of Stepwise Structuring of Cash Flow Analysis Within Spreadsheets Many might ask why go through the process of using two equations, i.e., the single payment compound amount formula and the single payment PV formula in a stepwise manner as was done above to obtain $39,499. Instead, they might ask couldn’t you compress these two formulas to one formula and thus use less steps in the spreadsheet model. It is of course possible to compress these two equations into one equation. The result is presented as follows:  n ð1 + inflation rateÞ (3:24) P=F ð1 + nominal discount rateÞ

3.6 Fundamental Concepts of Finance

81

Applying this equation yields the same result of $39,499 as demonstrated further:   ð1 + 0.03Þ 5 = $39,499 P = $50,000 ð1 + 0.08Þ Although equation (3.24) yields the same result as the two processes described earlier, there are two advantages associated with employing the stepwise approach when developing a cash flow model within a spreadsheet environment. The first advantage is that the stepwise method facilitates the creation of output tables and graphs that focus on individual elements of the analysis. For example, the decision makers (or stakeholders) may wish to see an output table showing how much money they will be spending in real dollars in a given year or they may be interested in viewing a cumulative cost over time curve that accounts for inflation (i.e., nominal cumulative cost over time curve). If the model is structured in a way that avoids a step-wise process using more efficient or condensed equations such as equation (3.24), the information needed to produce the requested outputs is tied up in equations and not easily available for producing the desired output. To produce the requested outputs, the analyst will have to return to the model and restructure it in a step-wise fashion. Over the years, we have learned that the more stepwise and simplified the spreadsheet model, the greater the flexibility in producing the desired output results. The second advantage of the stepwise approach is that it increases transparency, i.e., it makes it easier for decision makers and stakeholders to review the model and understand what is happening in each row or cell and the effect of applying the various equations. In our experience, increasing transparency increases the trust and acceptance of the model results. 3.6.2.4 Calculating NPV Now that we have defined and described the single payment PV formula, single payment compound amount formula, and appropriate interest rates, we are ready to define NPV and a simple way to calculate it within a spreadsheet environment. Simply stated, NPV is the difference between the PV of cash inflows and cash outflows over a period of time. Note the important word that it is the “difference” between the PV inflows and outflows. We have seen numerous cash flow analysis spreadsheets that deal only with costs (as is often the case with environmental remediation projects), where the developer of the analysis will refer to the sum of the PV costs as the NPV. The term NPV is not appropriate since there is no difference of inflows or outflows involved. The proper term for such an analysis is the PV cost. For those wishing to indicate that the PV involves the sum of PV cost from many years of operation, the proper term is total PV cost. NPV is an investment criterion developed for the purpose of evaluating investment opportunities. In general, capital investment opportunities such as the construction of a manufacturing plant or office building require heavy capital investment expenditures

82

3 Foundations of MCDM

(CapEx) early on (sometimes extending over several years to complete construction) followed by revenue from the selling of products or the rental of space. In addition, operating cost will be incurred during all years of operation. Converting all revenues and costs to PVs allows for their summation. The basic NPV rule is that only projects with positive NPVs are worth investing since they are worth more than they cost. Of course, managers working within a large corporation may have many competing opportunities to invest in and so they may choose projects with higher NPVs over those with lower NPVs. In most cases, the analysis does not stop there as the managers will consider other investment metrics such as the internal rate of return or payback period of competing projects. However, our focus here is not on capital budgeting but rather a simple and transparent way to calculate NPV within a spreadsheet environment. Simply stated, NPV can be easily determined by first estimating the year when each cash flow, i.e., CapEx, OpEx, or revenue will occur. Within the MCDM template, this year of such expenditures is determined probabilistically. Revenue cash flows are, of course, represented by positive numbers and expenditures are represented as negative values. These values are then summed with the year that they occur. Next the yearly sums are inflated i.e., converted to nominal dollars using equation (3.21), the single payment compound value formula (step 1). Next, the total nominal dollars associated with each year (regardless of being positive of negative) is converted to a PV using equation (3.20), the single payment PV formula (step 2). Once this is done, the present values from all years can be summed to provide the NPV. This process, including screen shots of an example spreadsheet model, is provided in Section 5.8. In addition, the structure can be reviewed in the MCDM Template provided along with this book. In the case where alternatives are being compared that involve only costs and no revenue, as is the case with the sediment project included in our case study, we recommend that the costs be signed as positive values. This makes the various output graphs and tables easier to review and understand. And, as previously stated, these costs should be reported as PVs since there is no “net” involved.

3.7 Fundamental Concepts of Economics Economics is a broad study and difficult to do justice in a brief overview. However, like our other subject areas, only a few of the most fundamental concepts need to be reviewed regarding their application to MCDM. At the outset it should be noted that MCDM is not concerned with macroeconomics which deals with overall economic behavior and issues of inflation, unemployment, and economic growth. Rather, it’s the study of microeconomics, which focuses on the behavior of individual economic decision makers such as consumers, workers, corporations, and business managers that is most applicable to MCDM.

3.7 Fundamental Concepts of Economics

83

3.7.1 The Basic Economic Problem Whenever the subject of economics is raised by politicians, the media, employers, employees, and activist groups (to name only a few) there are many issues that are identified. These include issues of inflation, unemployment, recessions, wages, interest rates, and budget deficits. With these topics in mind, which are the most important? Put another way, what is the most basic economic problem? Economist have long identified that the basic problem of economics is scarcity. This problem exists because human wants are unlimited, and resources are limited. Human wants include all the goods and services that humans desire including food, clothing, shelter, transportation, entertainment, and anything else enhances the overall quality of life. Regardless of how well an individual’s needs are met, there is often still more that could be done to enhance one’s overall quality of life. Resources are limited because there is only so much raw material, labor, equipment, energy, time, and talent needed to produce the various goods and services that humans desire. Some might say that for extremely wealthy individuals, scarcity is not a problem since such individuals have all the resources that they need. However, even the ultrawealthy are constrained in terms of things like time and health. Money cannot extend the number of hours in the day and, even with the best of health care, there’s only so much that can be done to extend one’s life. Therefore, even ultrawealthy individuals are constrained and are forced to make choices. The bottom line is that the scarcity of resources means that we are constrained in our choices regarding the goods and services we will produce and about the human wants that we will be able to satisfy. Therefore, economics is often described as the science of constrained choice [27].

3.7.2 Opportunity Cost The concept of scarcity by its very nature implies that choices must be made in terms of how best an individual, business, or group should go about meeting their needs. This leads to what may be the most fundamental underlying concept of economics; the concept of opportunity cost. Simply stated, opportunity cost is the value of the alternative that is sacrificed whenever a choice is made. It does not matter if this choice is made by an individual, business, or group. Opportunity cost, as the name suggests, means that all choices involve trade-offs. In Chapter 1 we’ve already defined trade-offs as giving up something valued to gain more of something valued higher. Therefore, the choices that we make provide indicators and insights into what we value.

84

3 Foundations of MCDM

3.7.3 Rational Person Assumption One of the assumptions of economics is that individuals behave rationally, meaning that they have certain goals and objectives and will pursue these goals in a rational manner. Thus, when making decisions individuals will seek alternatives that make them better off and avoid those that make them worse off. Therefore, it can be said that rational people pursue their own self-interest. In seeking their own self-interest, rational people respond to incentives. An incentive is anything that changes the benefit or cost associated with an action. As individuals seek their own self-interest and respond to incentives, economists like to say that the individuals are seeking to maximize their objective function. This objective function for the most part is theoretical in nature and would be very difficult to state in mathematical terms for most individuals without knowing a great deal about them and the choices that they make. However, with the advent internet shopping, social media, and machine learning, savy tech companies are learning more and more about the objective functions of the individuals that interact with these services. It should be noted that the MCDM process involves a series of steps that are used to develop an objective function that reflects the values, objectives, and preferences of decision makers and stakeholder groups.

3.7.4 Revealed Preference Analysis Revealed preferences is a type of trade-off analysis that seeks to understand the preferences, objectives, and values of individuals based on the choices that they make. It is most often used to understand the preferences of individuals for products and the prices that they are willing to pay for certain product features. The process involves statistical analysis and methods such as linear regressions, probability trees, neural networks, and Bayes’ Formula to identify the attributes and the weights that individuals are placing on the various attributes to make their product selection. Companies like Amazon and Netflix use these methods to suggest books, products, movies, and streaming series that individuals may be most interested. Revealed preference analysis can be used to evaluate the product features that interest most individuals. It can also be used to identify other elements of an individual’s objective function such as the value they place on living in certain neighborhoods, recreational activities, and natural environments.

3.7.5 Stated Preference Analysis Stated preference analysis is like revealed preference analysis in that it seeks to understand the preferences, objectives, and values of individuals based on the choices they

3.8 Behavioral Economics

85

make. The difference is that instead of waiting for individuals to make choices in the form of a purchase or an action, such as visiting a park, stated preference analysis makes use of survey techniques that require the individual to make subjective trade-offs regarding competing alternatives each scoring differently (i.e., producing different levels of outcomes) regarding the parameters that those taking the survey may care about. The MCDM process we describe in this book makes use of conjoint surveys to conduct stated preference analysis. These surveys are used to determine the willingness of decision makers and stakeholders to make trade-offs among a set of potential outcomes (also referred to as impacts or consequences). The trade-offs that are made are analyzed to determine preferences which are then quantified in the form of criteria weights.

3.8 Behavioral Economics As previously mentioned, economics assumes that individuals seek their own best interest and that they will rationally make decisions that will make them better off and avoid those that make them worse off. This is probably best described as the view of classical economics. However, behavioral economics which has its origins in the work on uncertainty and risk by Israeli psychologists Amos Tversky and Daniel Kahneman in the 1970s and 1980s demonstrated that this is often not the case. In a paper published in 1984 by the American psychologist, Kahneman and Tversky focused on issues of normative analysis and descriptive analysis in decision making. Normative analysis is concerned with the nature of rationality and the logic of decision making [28]. In other words, normative analysis is concerned with how we should make decisions. Descriptive analysis, in contrast, is focused with people’s beliefs and preferences as they are and not as they should be [29]. As noted in this paper, the tension between normative and descriptive consideration characterizes much of the study of judgment and choice. From our perspective, this tension characterizes much of the work of behavior economics. Although the work of Tversky and Kahneman is far reaching in terms of the ways people make decisions, we review just a few very interesting results of their research. In Choices, Values, and Frames, Kahneman and Tversky demonstrate that: – When it comes to decision involving uncertain gains, people are risk-averse. – When faced with decisions involving uncertain losses, people are risk-seeking. – When decision problems are framed in different ways, people will change their preferred choice, even though the underlying decision is the same. Each of these results are not what would be expected if people were purely rational and using normative decision processes. Using three of the examples given in Choices, Values, and Frames, we demonstrate the three noted results. The three examples are adapted with permission. Copyright © 1984 by American Psychological Association Kahneman, D., Tversky, A., Choices, Values, and Frames, American Psychologist, American

86

3 Foundations of MCDM

Psychological Association, 1984, 39:4. 341–350. However, instead of just repeating the examples described in the paper, we make use of the PrecisionTree program to provide a graphical view of the examples that Kahneman and Tversky describe using only text. It should be noted that although the examples provided can seem rather simplistic they reveal a great deal regarding basic attitudes toward risk and value.

3.8.1 Risk Aversion in Gains We begin with a simple coin toss involving a $10 wager. The wager is simply that if the coin results in heads, the individual taking with wager wins $10 and if the coin results in tails, $10 is lost. Of course, when confronted with this wager, the individual can choose to simply not take the wager at all. Figure 3.19 presents a decision tree view of this simple wager.

50.0%

Heads

$10 Equivalent EV

Yes

50.0% $10

Outcome

$0

0 50.0%

Tails

-10

50.0% -10

Wager

Coin Toss Wager

0 No

Equivalent EV $0

100.0% $0

Figure 3.19: Risk neutral coin toss wager.

Note that in Figure 3.19, the expected value of the coin toss outcome is zero and the return of not entering the wager is also zero. Based on these results an individual should be indifferent to making the wager since both outcomes are equivalent (i.e., from an expected value point of view). This is especially true if the resulting outcome involves merely a change of wealth and not a change of the overall state of wealth of the individual (i.e., results of the wager will have no impact whatsoever on the individual’s wellbeing or lifestyle). However, most individuals are unwilling to make this wager. The reason as presented by Kahneman and Tversky is that the attractiveness of the possible gain does not outweigh the aversion to the possible loss. In their paper, Kahneman and Tversky indicate that most respondents in a sample of undergraduates refused to stake $10 on the toss of a coin if they stood to gain less than $30. Figure 3.20 shows the decision tree for this wager.

3.8 Behavioral Economics

50.0%

Heads

$30 TRUE

Yes

87

50.0% $30

Outcome

0

$10 50.0%

Tails

-$10

50.0% -$10

Wager

Increased Reward Coin Toss

$10 No

FALSE 0

0.0% 0

Figure 3.20: Increased reward of coin toss wager.

Note in this solved tree the expected value of the risk wager is now $10. One way to think about taking this wager is that an individual taking this wager has increased their expected value gain by $10. In accordance with the expected value theory, a rational individual would make this wager (if a negative outcome would not affect their overall wealth status). Of course, at the end of the coin toss they will either win $30 or lose $10. An individual who would not make this wager could be assumed to be highly risk-averse in uncertain gains. We have seen risk aversion in gains in actual practice on many occasions. An occasion that comes to mind involved presenting such a decision tree to a lawyer faced with deciding whether to proceed to trial or accept a settlement. Although the probability of winning the lawsuit and the potential dollar value of the win was more than sufficient to proceed with the court case, the lawyer chose settlement saying: “As I understand expected value theory, the lawsuit is the right thing to do if I were going to be playing this game many times. However, I’m only going to be playing it once and I do not wish to lose.”

3.8.2 Risk-Seeking in Losses Our next example is a demonstration of risk-seeking regarding a situation involving uncertain loss. Again, we will use a decision tree to demonstrate one of the examples from Kahneman and Tversky’s paper on Choices, Values, and Frames. In this example, the situation is one where the individual is forced to choose between a sure loss of $800 or an alternative that involves an 85% chance losing $1,000 and a 15% chance of losing nothing. Figure 3.21 presents this situation.

88

3 Foundations of MCDM

85.0%

Loss

-$1,000 Risk Alternative

FALSE

Risk Event

0

-850 15.0%

No Loss

$0 Loss Choices

0.0% -$1,000

0.0% $0

Loss Decision -800 Sure Loss

TRUE

100.0%

-$800

-$800

Figure 3.21: Decision tree regarding competing loss choices.

The PrecisionTree program has solved for this situation based on expected value theory and the results indicate that one should choose to accept the sure loss. The reason is that the risk alternative has an expected value loss of $850 which is greater than the sure loss of $800. However, Kahneman and Tversky report that when faced with a situation such as this, a large majority of people express a preference for a gamble over a sure loss. In other words, most people are risk-seeking in losses. One might wonder what type of situation would arise whereby an individual only has a choice between a sure loss or a potential larger loss. This is not as uncommon as one might expect. One such situation is where a defendant in a court case must decide whether to offer a settlement amount versus preceding to court. Of course, the dollar value associated with most court cases would be substantially larger than those presented in Figure 3.21.

3.8.3 Choice Preference as a Function of Decision Frame Another interesting result of Kahneman and Tversky’s work is that preferences for certain choices are subject to change based on the way that a decision problem is framed. According to Kahneman and Tversky: All analysis of rational choice incorporate two principals: dominance and invariance. Dominance demands that if prospect A is at least as good as prospect B in every respect and better than B in at least one respect, then A should be preferred to B. Invariance requires that the preference order between prospects should not depend on the manner in which they are described. In particular, two versions of a choice problem that are recognized to be equivalent when shown together should elicit the same preference when shown separately [29].

In their paper, Kahneman and Tversky demonstrate that invariance, although a seemingly elemental and innocuous requirement, cannot generally be satisfied [30].

3.8 Behavioral Economics

89

In their paper, Kahneman and Tversky present the same decision problem to two different groups except that they describe the problem differently. The decision problem was described to the first group, which included a total of 152 respondents, that they were to imagine that the United States is preparing for an outbreak of an unusual disease which is expected to kill 600 people. There are two alternative programs that have been developed to combat the disease and that the exact scientific estimates of the consequences are as follows: – If Program A is adopted, 200 people will be saved. – If Program B is adopted, there is a one-third probability that 600 people will be saved and a two-third probability that no one will be saved. The group was then asked to indicate which program they favored. The results of the first group were: 72% chose Program A and 28% chose Program B. For a second group of individuals, which included 155 respondents, the same cover story was provided but the program descriptions were changed as follows: – If Program C is adopted, 400 people will die. – If Program D is adopted, there is a one-third probability that nobody will die and a two-third probability that 600 people will die. In the case of the second group, 22% selected Alternative C and 78% selected Alternative D. The difference of choices among the two groups represents an astounding result. This is because the choices are the same, only described differently. A close review of the descriptions reveals that Programs A and C are the same and Programs B and D are the same. Therefore, if individuals were making purely rational choices (normative decisions) then the percentages for the chosen programs should be roughly similar. Figure 3.22 provides the decision tree that is representative of the program descriptions associated with this exercise. Note the program choices have been named to indicate the equivalence of Programs A and C and of B and D. Furthermore, note that the expected number of individuals saved under Programs B and D is the same as the sure outcome of 200 saved under Programs A and C. One would expect the same percent of respondents to choose Program A and C, because they yield the same certain outcome. Similarly, the same percent of respondents should choose B as D, because they yield the same expected outcome. The selection of Program A (or C) results in an outcome where 400 will certainly die. However, the description for Program A frames this outcome from the standpoint that 200 people will be saved. The description of Program C on the other hand frames this outcome in terms 400 people will indeed die. The bottom line is that when framed in terms of lives saved the respondents wish to be risk-averse (consistent with the findings of risk aversion in gains). However, when framed in terms of lives lost people seek to be risk-seeking (consistent with the findings of risk-seeking in losses).

90

3 Foundations of MCDM

All

Program B & D

Equivalent EV

33%

33%

600

600

67%

67%

People Saved

0

200 None

0 Decision Frame Example

0

Program 200 Programs A & C

Equivalent EV 0

100.0% 200

Figure 3.22: Decision tree for alternate framing example.

Kahneman and Tversky report that sophisticated respondents, even when participating in the two experiments within minutes of each other, will display this risk aversion or risk-seeking depending on the frame. However, when confronted with their inconsistency they are often puzzled. Even after rereading the problems, they will want to maintain their risk-aversion or risk-seeking depending on the frame, and yet, they also want to provide consistent answers in the two versions that is, to maintain invariance [30]. The examples indicate that individuals do not always follow the rationality assumptions that underlie economic theory. There are other forces at play in terms of our perceptions and perhaps values that alter the way decisions are made. The last example regarding the change in decision based on how the problem is framed has implications for MCDM. As we will see in Chapter 4, a significant part of the MCDM process involves developing a proper framing of the problem, i.e., one draws out the various issues of the problem and makes them transparent to the decision makers and stakeholders.

3.8.4 Cognitive Biases The issue of cognitive biases was first raised in Chapter 1 regarding the advocacy-based approach (see Section 1.2.3.1). In that section, a cognitive bias was defined as a systematic error in thinking that occurs when people are processing and interpreting information in the world around which affects the decisions and judgments that they make [31]. The issue of cognitive biases is a major area of study for behavioral economists who are seeking to understand how people make decisions and why they often go wrong and ways of improving decision making. There are many, many cognitive biases. As noted in Chapter 1, at the time of this writing Wikipedia List of Cognitive biases includes a total of 188 cognitive biases. Its not useful to review all the different cognitive biases here. In Section 1.2.3.1, we provide a summary of some of the most common cognitive biases.

3.8 Behavioral Economics

91

Risk aversion and risk-seeking are not necessarily cognitive biases as they can be more of a personality trait. However, there is a cognitive bias known as the ambiguity effect which may be closely related to risk aversion. The ambiguity effect is a type of bias whereby people prefer a known outcome rather than taking a chance. The example involving a change in decision based on how the problem is framed is a type of cognitive bias known as the framing effect, which is a bias where people decide on options based on whether they are presented with positive or negative connotations. One way to understand the causes and impacts of cognitive bias is through the concepts of System 1 and System 2 thinking. According to Kahneman and Tversky, these two systems are mechanisms by which we evaluate and react to the world around us. System 1 is thinking fast or intuitively and requires little mental effort to initiate. System 2 is thinking slow or deliberatively and requires an effort to initiate and use. System 1 is based on our acquired experience and uses that information quickly and effortlessly. System 1 is why we do not need MCDM for many decisions, such as what to have for breakfast at your favorite restaurant, what to do at a green light, or how to interpret a smile. In a business setting, our experience may tell us how best to work with water quality regulators from California or the costs and benefits of building trails near a marine environment. System 2 is used for evaluating more difficult and unfamiliar tasks and situations. It is for situations that require conscious effort and deliberate choices such as what to have for breakfast at a vegan restaurant, if you love bacon and sausage or what to do at a light that is flashing green and red. In a business setting, it might design a strategy to work with water quality regulators from New York or assessing the value of riverwalks in an urban setting. Typically, the two systems work well together and “assign” a decision to the appropriate system. However, System 1 cannot be turned off and is subject to biases and System 2 is sometimes reluctant to be engaged because of the effort involved. As a result, decisions may not be optimal. We may assume that dealing with water quality regulators from New York is the same as dealing with the ones from California, and that the value urban riverwalks are basically the same as coastal trails. Thus, the goal of MCDM is to facilitate the engagement of System 2 to make thoughtful, deliberate decisions. MCDM is more useful when it helps avoid the biases of System 1, “what my gut tells me” decision making. In terms of MCDM, the most important thing regarding cognitive biases is recognizing that they exist and working to remove them as best as possible. Fortunately, the mere recognition that one may be engaging in a cognitive bias goes a long way in eliminating the bias. In Section 5.4 of this book, we present an exercise for reducing cognitive biases that often come into effect when estimating cost ranges.

92

3 Foundations of MCDM

3.8.5 Emotions and Rationality When first introduced to MCDM, or any structured decision analysis process for that matter, a common assumption that many people make is that the goal is to remove emotions from the decision process and to state everything in mathematical terms and focus on the cold hard facts. Perhaps this impression comes about because in many situations the often-heard refrain is “just show me the numbers.” In fact, it is a common assumption, and the advice of many, that good decisions come from cold, hard, rational analysis, especially when it comes to business. The notion of removing emotion from business decisions is reinforced even in popular culture. For example, one of the most famous lines from the movie The Godfather occurs just after Michael Corleone declares that he will kill the two men who attempted to assassinate his father and take over the family business. When his brother Sonny, now acting as “Godfather” while their father recovers says to Michael “your taking things way too personally,” Michael responds, “It’s not personal Sonny, it’s strictly business.” Given this background it would seem there is no room for emotions in business or rational decision making. However, is this true? The answer is no. In his book Descartes Error – Emotions, Reason, and the Human Brain, Antonio Damasio reports on the results of over two decades of working with and studying the history of individuals who experienced traumatic brain damage to the frontal lobe tissue as a result of physical injury or disease. The frontal lobe region of the brain is the area that has been found to govern functions such as emotions, impulse control, and social interactions. In working on an early case with an individual who had this type of injury, Damasio reports that [32]: I had before my eyes the coolest, least emotional, intelligent human being one might imagine, and yet his practical reason was so impaired that it produced, in the wanderings of his daily life, a succession of mistakes, a perpetual violation of what would be considered socially appropriate and personally advantages [32]

Damasio goes on to state of this patient [33]: The instruments usually considered necessary and sufficient for rational behavior were intact in him [the patient]. He had the requisite knowledge, attention, and memory; his language was flawless; he could perform calculations he could tackle the logic of an abstract problem. There was only one significant accompaniment to his decision-making failure: a marked alteration to experience feelings.

This observation suggested to Damasio that “feeling was an integral part of the machinery of reason” [34]. Damasio reports that through two decades of clinical and experimental work with a large number of neurological patients allowed him to replicate this observation. The work of Damasio is far reaching and has implications for decision analysis in general and perhaps MCDM specially. Rather than go into all the details of the

3.9 Decision Quality

93

book, which we suggest that anyone interested in decision analysis and the psychology of decision making should read; some of the more important lessons from Domasio’s work as they pertain to MCDM are: – Certain aspects of emotions and feelings are indispensable for rationality. – Emotions and feelings assist us in predicting uncertain futures and planning our actions accordingly. – Emotions felt in the body, referred to as somatic markers by Damasio [35], help us to predict the negative outcomes of certain alternatives and reduce our options. – Emotions and feelings may remove some of the complexity from a problem so that the tools of logic and reason may then be applied. The result of all of this is that we should not be so quick to think we need to remove emotions and feelings from our decision making. In addition, our gut reactions can indeed be informative. This is not to say that we should only go with our gut reactions. As Damasio states in the introduction to Descartes’ Error, this is not to deny that emotions and feelings can cause havoc in the process of reasoning in certain situations. However, the absence of reason and feeling is no less damaging [36]. One important goal of MCDM process is to allow stakeholders and decision makers to openly evaluate the proper role of emotions and rationality. In the previous section, we learned that cognitive biases exist which can lead to errors in our thinking. Therefore, we need to identify them and manage them. It is possible that many of these cognitive biases are the result of gut reactions. On the other hand, we’ve learned that emotions and feelings are important in assisting us with rational thinking and the analysis of our decision alternatives. Regarding MCDM we believe that the process of identifying values, objectives, and preferences incorporates our emotions and feelings and helps to simplify our problem, identify doable alternatives, which can then be analyzed to find the alternative that provides the greatest value.

3.9 Decision Quality Before ending this chapter on the foundations of MCDM it is important to review what is arguably the most fundamental concept of all regarding decision analysis and that is the concept of decision quality. This is because the whole point of decision analysis is to make a good decision, which as defined in Chapter 1, is one that is logically consistent with our preferences for potential outcomes, our alternatives, and our assessment of uncertainties [36]. But how would we know if we’ve indeed made a good decision. One method of doing so is by viewing the decision through the lens of the decision quality chain presented in Figure 3.23. The decision quality chain has been used applied in the field of decision consulting and embedded in the literature since at least the late 1990s. The concept of the decision quality chain was first developed by David Matheson and Jim Matheson and appeared

94

3 Foundations of MCDM

2. Creative, Doable Alternatives

3. Meaningful, Reliable Information

Elements of Decision Quality 1. Appropriate Frame

4. Clear Values and Trade-Offs

5. Logically correct Reasoning

6. Commitment to Action

Figure 3.23: The decision quality chain. The rights to the Decision Quality image is owned by SmartOrg and is used here with permission granted by David Matheson and Jim Matheson, founders of SmartOrg.

in their book The Smart Organization published in 1998. According to Matheson and Matheson the overall quality of a decision can be summarized in six dimensions as shown in Figure 3.23 [37]. The decision must have good quality in all six dimensions, or a good decision has not been made. Note that each of the dimensions are presented as a link in a chain. This supports the notion that for a good decision to have been made it must fare well in all dimensions since it is well known that a chain is only as strong as its weakest link. The six dimensions of decision quality are briefly described further.

3.9.1 Appropriate Frame The basic idea of the frame is that it is focused on the question of whether we are solving the right problem. As Matheson and Matheson suggest that because the frame is the “window’” through which we view the problem, it is the hardest dimension to see (note there is more discussion of the project frame in Section 4.5.1 in relation to the decision hierarchy). The project frame is primary focused on understanding the purpose of the project and its scope and perspective of the decision makers and stakeholders involved.

3.9 Decision Quality

95

3.9.2 Creative Doable Alternatives For any decision to take place, there must be alternatives to be decided upon. This dimension is focused on answering the questions regarding whether the alternatives are implementable and address the issues identified in the project frame. In addition, have they been fully evaluated in terms of addressing the purpose of the project.

3.9.3 Meaningful Reliable Information This dimension is focused on the quality and reliability of the information that is being used to evaluate the decision alternatives. This is the dimension that addresses the concern, or better stated, avoids the problem of garbage in, garbage out.

3.9.4 Clear Values and Trade-Offs This is the dimension that asks, have we described our values, as well the objectives that support them and ultimately criteria that can be used to score alternatives? In addition, it asks, have we identified our preferences and our willingness to make trade-offs among the criteria?

3.9.5 Logical Correct Reasoning This dimension is focused on whether we have developed a representative model for evaluating alternatives. This is one that properly relates the various decision elements and provides output results that enable us to logically evaluate alternatives.

3.9.6 Commitment to Action Simply stated, the best decision in the world is useless if it is not implemented. If there is no commitment to action, there is no point in entering into the decision analysis process in the first place. According to Matheson and Matheson [38]: In most cases the commitment to act is attained by involving the right people in the decision effort. The right people must include individuals who have the authority and resources to commit to the decision and make it stick (the decision makers) and those who will be asked to execute the decided-upon actions (the implementers).

96

3 Foundations of MCDM

References [1] [2] [3]

[4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30]

Behavioral Economics Explained. Wityniski, M., University of Chicago News, (accessed April 9, 2022 at https://news.uchicago.edu/explainer/what-is-behavioral-economics). Ibid. Faculty Profile Antonio Damasio. University of Southern California Dornsife, College of Letters, Arts and Sciences Accessed (accessed April 10, 2022 at https://dornsife.usc.edu/cf/faculty-and-staff/fac ulty.cfm?pid=1008328). Project Management Institute, A guide to the project management body of knowledge, sixth edition, Project Management Institute, Inc., Newtown Square, PA, USA, 2017, pp. 179. Denardo, E.V., The science of decision making, A problem-based approach to using Excel, New York, NY, USA, John Wiley & Sons, Inc. 2002, p. 218. Ibid. pp. 245. Ibid. pp. 244. Ibid. About Systems Engineering: International Council on Systems Engineering (Accessed April 11, 2022 at https://www.incose.org/about-systems-engineering/about-systems-engineering). Seila, A. F., Ceric, V., & Tadikamalla, P., Applied Simulation Modeling. Belmont, Belmont, CA, USA Thomson Brooks/ Cole, 2003, pp 2. Lindeburgh, M. R., Engineer-in-Training Reference Manual, Eight Edition, Professional Publications, Inc., Belmont, CA, USA, 1992, p. 22–1. Fuller, R. B., Synergetics, Explorations into the Geometry of Thinking. McMillian, New York, NY, USA, 1975, pp. 95. Edmondson, A. C., A Fuller explanation, The synergetic geometry of R. Buckminster Fuller, Cambridge, MA, USA, Birkhauser Boston, Inc., 1987, pp. 31. Ibid, pp. 32. Buckminster Fuller: The Planet’s Friendly Genius, University of Chicago MAROON, May 24, 1981. Havranek, T. J., Sustainable Remediation Panel, Remediation, Winter 2011, pp. 137–140. Fuller, R. B., & Applewhite, E. J., Synergetics dictionary, The mind of Buckminster Fuller, Volume 4. New York, NY, USA,: Garland Publishing, Inc., 1985. pp. 101. Clemen, R. (1990). Making hard decisions: An introduction to decision analysis. Belmont, CA, USA Duxbury Press, 19990, pp. 169. Lindeburgh, M. R., Engineer-in-Training Reference Manual, Eight Edition, Professional Publications, Inc., Belmont, CA, USA, 1992, p. 11–3. Newendorp, P., & Schuyler, J. (2014). Decision analysis for petroleum exploration. Aurora, CO, USA Planning Press, 2014, pp. 156. Vose, D., Risk analysis, A quantitative guide (2nd ed.), New York, NY, USA, John Wiley & Sons, Inc., 2000, pp. 41 Ibid. pp. 41. Ibid. pp. 61. Ibid. pp. 61. Ibid. pp. 100. Ibid. pp. 102. Besanko, D. A., Braeutigam, R.R., Microeconomics, An integrated approach, New York, NY, USA, John Wiley & Sons, Inc., 2002, pp. 3. Kahneman, D., Tversky, A., Choices, Values, and Frames, American Psychologist, American Psychological Association, 1984, 39:4,341–50. Ibid. pp. 344. Ibid. pp. 345.

References

[31] [32] [33] [34] [35] [36] [37] [38]

97

Cherry, K. What Is Cognitive Bias. (Accessed from verywellmind, January 10, 2020 at https://www. verywellmind.com/what-is-a-cognitive-bias-2794963). Damasio, A., Descartes’ error, Emotion, reason, and the human brain, New York, New York, USA, Penguin Books, 2005, 25th printing, pp. xv. Ibid. pp. xvi. Ibid. Ibid. pp 173. Parnell, G. S., Bresnick, T. A., Tani, S. N., & Johnson, E. R., Handbook of Decision Analysis, John Wiley & Sons, Inc. Hoboken, NJ, USA, 2013, pp. 3. Matheson, E., Matheson, J., The smart organization, Boston, MA, USA, Harvard Business School Press, 1998. Matheson, E., Matheson, J., The smart organization, Boston, MA, USA, Harvard Business School Press, 1998. pp. 16.

4 The MCDM Process The MCDM process is designed to create a collaborative journey of inquiry with the destination of finding the best, highest value, strategic alternative (or design) for addressing a particular issue or technical problem. Although there are many ways that a collaborative group could undertake such a journey, this chapter provides a systematic structured process that we’ve found to be particularly effective. The process we present here is an amalgamation of processes developed by other authors (see [1–3]) and best practices resulting from our experiences in providing decision analysis services to clients both individually and in collaboration with other decision analysts. When implemented as described, this process will result in a good decision as defined in Section 1.5.4 by addressing the six elements of decision quality described in Section 3.9. The use of the complete process is based on the assumption that the decision, project, or issue faced by decision makers/stakeholders is of sufficient scale and impact to warrant stochastic MCDM. A list of attributes of such decisions/projects/issues is provided below. Information regarding these attributes is provided in the paragraphs following this list: – High stakes – Numerous stakeholders – Multiple and conflicting objectives – Numerous and complex alternatives – Significant risks and uncertainties – Long-range impacts “High stakes” is a relative term depending on the size of the business or organization faced with the decision problem. A $5 million investment decision might be high stakes for a small business or local government, whereas such a decision might be seen as low stakes by a large corporation. In general, we envision that the process provided here would be used on projects involving costs ranging from tens of millions of dollars to upward of hundreds of millions of dollars. As the number of stakeholders increases, the need for a structured decision process increases. This is true even when the investment needed for a given project may be relatively low. It is especially true whenever influential adversarial stakeholders are present who may seek to prevent a decision from being implemented. In other cases, due to a misunderstandings or mistrust, adversarial stakeholders may advocate for alternatives that, if implemented, would lead to outcomes that are detrimental to their objectives and interests. Whenever decision makers/stakeholders seek to consider nonfinancial objectives, such as those associated with the use of the integrated capitals approach (or the use of environmental, social, and governance (ESG) metrics) the need for a structured decision process increases. This is because as the number of objectives increases, the https://doi.org/10.1515/9783110765861-004

4.1 Is the Complete MCDM Process Required for Every Decision?

99

number of criteria (i.e., value measures) used to evaluate how well various alternatives will contribute to such objectives also increases. As a result, a process is needed for determining the relative importance of each of these value measures and the willingness to make trade-offs among them. As the number of objectives increases, the potential for conflicting objectives also increases, further increasing the need for a structured decision process. The number and complexity of alternatives typically increases as the number of strategic decisions increases and as the number of choices associated with each strategic decision increases. It is often the realization that there are many strategic decisions that must be made and that the choices associated with the strategic decisions are interrelated and will interact in complex ways that leads decision makers and stakeholders to realize that a structured decision process is needed. The structured process provides a significant value whenever there are many risks and uncertainties associated with a given project. Recall that risk is defined as an uncertain condition that, if it occurs, will have either a positive or negative impact on a project’s outcomes. The MCDM process helps to increase the likelihood of capitalizing on or capturing positive risks (i.e., opportunities). In addition, the process can be used to identify alternatives that avoid certain negative risks entirely or to reduce the probability they occur and/or their impact. The long-range impact of a decision is an attribute that gives rise to the need for a structured decision process. It’s easy to imagine that a new highway, manufacturing plant, large-scale distribution center, or residential development will have impacts ranging far into the future. The longer and larger the potential impact, the more the need for structured decision process.

4.1 Is the Complete MCDM Process Required for Every Decision? There are many business managers or government officials who would say that all their decisions involve one or more of the attributes listed earlier. Therefore, they may ask, “does this mean that the complete process is required for all their decisions?” The short answer is no. However, such decision makers may find that portions of the complete process are appropriate depending on the nature and complexity of the decision under consideration. Figure 4.1 is from an article by Ralph Keeney titled Making Better Decision Makers published in the December 2004 issue of the journal Decision Analysis [4]. In this article, Keeney provides a prescriptive approach regarding how a hypothetical set of 10,000 decisions should be made by individuals trained in the concepts of decision analysis. Keeney indicates that of the 10,000 decisions, perhaps 9000, or 90%, have consequences that are either too small to be of concern or have obvious solutions (socalled no brainers). The remaining 1,000 decisions, or 10%, are therefore worthy of

100

4 The MCDM Process

A Prescription for How 10,000 Decisions Should be Resolved

750 Resolved by Clear Thinking Consistent with Decision Analysis

All Decisions 10,000

2,000 No Brainers

Worth Thinking About 1,000

Get Appropriate Systematic Thought 1,000

40 Resolved by Clarifying Problem

200 Resolved by Partial Decision Analysis

Resolved by Complete Decision Analysis 50

40 Resolved by Creating Alternatives

20 Resolved by Making Trade-offs

Resolved by Addressing Risk Tolerance 5

Resolved by Resolved by Addressing Clarifying Linked Objectives Resolved by Decisions 40 5 Describing ConsequencesResolved by Addressing 30 Uncertainties 20

Small Consequences 7,000

Figure 4.1: A suggested prescription for resolving decisions. (Reprinted by permission, Keeney RL, Making better decision makers, Decision Analysis, volume 1, number 4, pp. 193–204. 2004. Copyright 2004, the Institute for Operations Research and the Management Sciences (INFORMS), 7240 Parkway Drive, Suite 300, Hanover, MD 21076, USA.)

systematic thought. We believe that the attributes listed above are what place decisions in the category of worthy of systematic thought [4]. Figure 4.1 further illustrates that of the 1,000 decisions worthy of systematic thought, approximately 200, or 20%, might be resolved by partial decision analysis such as by clarifying the problem, clarifying objectives, creating alternatives, addressing risks, and making trade-offs. Each of these partial decision methods is included in this chapter as part of the complete MCDM process. In terms of the complete process, Figure 4.1 indicates that only 50, or 5%, of the 1,000 decisions worthy of systematic thought fall into this category. These are the decisions that we believe would benefit the most from stochastic MCDM. A point of note regarding Figure 4.1 is that it suggests that 750, or 75%, of the 1,000 decisions worthy of systematic thought can be resolved by clear thinking consistent with decision analysis. It should be stressed that this figure is a prescription for how decisions should be resolved assuming that the decision makers approaching these decisions have had sufficient training in decision analysis methods. We believe that for this to be true for such a large percentage of decisions, a culture of decision quality

4.3 Outline of the MCDM Process

101

would need to be in place in the organization where these decisions are occurring. Without such a culture, many of the decisions falling into this category might be better addressed by partial decision analysis or complete decision analysis facilitated by skilled decision analysts. Regarding the programmatic creation of a culture of decision quality, a good reference on this area is The Smart Organization by David Matheson and Jim Matheson.

4.2 How Much Effort Should Be Invested in the Decision Analysis Process? A fair question posed by decision makers is how much effort should be invested in the decision analysis process. According to Parnell et al. [5]: A useful rule-of-thumb is the one percent rule, which states one should be willing to spend 1% of the resources allocated in a decision to ensure that the choice is a good one. So, for example, when deciding on the purchase of a $1,000 household appliance, one should be willing to spend $10 to gather information that will improve the choice. By the same token, a company deciding on a $100 million investment should be willing to spend $1 million to ensure that the investment decision is well made.

4.3 Outline of the MCDM Process Figure 4.2 outlines the complete stochastic MCDM process. The process consists of three primary phases: structure, evaluation, and agreement. Each phase consists of three steps, for a total of nine steps to complete the process. Structure

Evaluation

Agreement

Frame the Decision

Perform Probabilistic Analysis

Develop Output Results

Develop the Objectives Hierarchy

Structure the Model

Communicate Insights

Design Alternatives

Quantify Preferences and Uncertainties

Commit to Implement

Figure 4.2: Outline of the MCDM process.

102

4 The MCDM Process

All of the process phases and their associated steps are important. However, some are more important than others. Of the phases, the structure phase is the most important. This phase sets the stage for the entire process. Its purpose is to fully define and describe the decision problem and create a shared understanding of issues. Without such definition, description, and understanding, there is little possibility of finding a high value solution. In terms of the process steps, the last step, Commit to Implement, is the most important. In fact, if, after going through the entire process, there is no commitment to implement, then there was no point in entering into the MCDM process in the first place. Without a commitment to implement, i.e., to make the decision, all efforts and resources spent on the process will have been wasted. The Commit to Implement step is so important that it creates the need for establishing a decision review board (DRB). The DRB comprises individuals necessary and sufficient for the decision to be made and successfully implemented. No structured decision analysis process should begin without creating a DRB. Individuals selected for the DRB must agree to do three things. First, they must agree to participate in several meetings of two to three hours in duration. Second, they must agree to fully engage with the process by reviewing meeting preread materials, showing up to meetings prepared, and actively participating in group discussion during the meetings. Third, the DRB must agree that if presented with a compelling alternative that is consistent with the values, objectives, and preferences of the decision makers with a strong business case, they will commit to implement. The required meetings are typically scheduled as follows: Start of the structure phase: The purpose of this meeting is to: – Set boundaries in terms of policies, constraints, or prior decisions that are beyond the scope of the analysis – Review background information and primary issues to be addressed – Plan the activities associated with this phase End of structure phase: For the purpose of reviewing of the results of the structure phase activities and deciding whether to: 1. Proceed to the evaluation phase 2. Recycle back through the structuring phase activities if they believe the process has resulted in insufficient problem definition, poor understanding of the issues, or an inadequate set of alternatives During the agreement phase: The DRB’s role during the agreement phase is to review the results from the evaluation phase and either commit to implement or ask for additional information or recycling through the evaluation phase. The goal of the DRB is to gain sufficient understanding of the alternative identified by the process to improve its implementation. Although the DRB may require additional analysis before approving of a given alternative, at this point in the process and based on their prior agreement, their role is not to simply avoid making a commitment to implement.

4.4 Inviting Stakeholders to Share in Decision Making

103

4.4 Inviting Stakeholders to Share in Decision Making Whenever MCDM is performed by a private or publicly owned company, the DRB typically consists of company employees only. Included in this group are representatives from various departments such as operations, legal, finance, real estate, environmental, and senior management. External stakeholders are not included in the decision making. However, the values, objectives, and preferences of external stakeholders are often considered in the decision-making process. This is because the managers of such companies understand that to obtain the necessary permits, licenses, leases, and regulatory approvals needed to implement their decisions, they will need the support of a host of external stakeholders. Otherwise, the external stakeholders can put up numerous roadblocks that will cause delays, increase costs, and possibly the result in abandoning implementation entirely. Most governments, government officials, and regulatory agencies understand that there is a need to consult the people affected by the decisions that they make. Although some governments and regulatory agencies may do this very well, in some instances, regulatory frameworks can be based on the “decide and defend” approach. For example, in the United States, the CERCLA process for the cleanup of hazardous waste sites provides for a 60-day public comment period at the completion of each major project phase, by which time the public and other special interest groups may feel that it’s too late for their input to be meaningful. For large complex problems, the time periods are too short for the voluminous information that these groups must process. This leaves the external stakeholders with the feeling that they are on the outside looking in and, in the case of environmental cleanup, causes them to push for the most expensive and costly alternatives without understanding the totality of consequences associated with that alternative. We believe that there are better ways to achieve stakeholder support beyond simply attempting to consider the values of external stakeholders in our internal decision-making process or falling back on the “decide and defend.” Instead, we believe a collaborative approach can and should be fostered, whereby the question is not whether to involve external stakeholders in the decision-making process, but rather at which phase or step they will be involved.

4.4.1 Potential Levels of Stakeholder Involvement The International Association for Public Participation (IAP2) has developed a spectrum of public participation for including stakeholders in a collaborative decisionmaking process. The spectrum includes five levels, with each level increasing the amount stakeholder involvement and impact. The IAP2 Spectrum of Public Participation is described here with permission from IAP2 copyright International Association for Public Participation www.iap2.org. The five levels as described by Randall Pearce are as follows [6]:

104

–

–

–

–

–

4 The MCDM Process

Inform – This level is recognized as the traditional or perhaps commonly used approach for stakeholder communications. This approach involves using communications tools such as fact sheets, websites, and displays. According to Pearce, the promise of this approach to stakeholders is simply “We will keep you informed.” Consult – This level makes use of communication techniques such as surveys, focus groups, and town hall meetings. “However, in addition to keeping stakeholders informed, the promise extends to ‘listen and acknowledge concerns and provide feedback on how the input provided influenced the decision.’” Involve – This level increases the stakeholder interaction and includes more indepth work such as workshops or deliberative forums. “In addition to keeping stakeholders informed and letting them know how their views shaped the decision, the organization promises to ensure stakeholder ‘concerns are directly reflected in the alternatives developed.’” Collaborate – This level involves a higher level of partnership between stakeholders and the decision makers. It includes the use of workshops and participatory decision-making forums such as the facilitated framing meeting described in Section 4.5. When involved at this level stakeholders work hand in hand with decision makers to “‘to give direct advice and innovate in formulating solutions’ that will be incorporated into the plan or to the ‘maximum extent possible.’” Empower – This level is called “empower” because the decision makers commit in advance to implement the design developed by the decision-maker/stakeholder partnership. This level involves the use of deliberative forums such as Citizen Juries or the use of ballots to determine selection of the final alternative based on stakeholder consensus.

4.4.2 Recommended Levels of Stakeholder Involvement The level of stakeholder involvement in most of our MCDM projects has been primarily at the Inform or Consult levels. The decision regarding the level of stakeholder involvement in these projects has been made by our clients and/or their resident DRBs. We believe that the DRBs, i.e., those with the power to make the decision and control the resources for implementing them, should be the ones to make decisions regarding the level of stakeholder involvement. That said, we do have recommendations regarding the levels that are appropriate to most applications of MCDM. Depending on the decision, either the Consult, Involve, or Collaborate level will be most appropriate. In other words, the levels and the two ends of the spectrum, in most cases, are not appropriate. The inform level, at the lowest end of the spectrum, is akin to the “decide and defend” approach which we believe often leads to feelings of mistrust and disempowerment among stakeholder groups. Unless mandated by a governmental or regulatory process, such as the CERCLA process, we believe that this level should be avoided.

4.5 Structure Phase

105

The empower level is beyond what most publicly owned businesses could promise. The corporate directors, executives, and managers of public companies have a fiduciary responsibility to act on behalf of the company’s stockholders. External stakeholders do not have this same responsibility; therefore, their values, objectives, and preferences, although important, may not take into account the financial wellbeing of the company. The same is true of private businesses whose owners, although willing to consult, involve, or collaborate with external stakeholders, would in most cases be unwilling to empower these stakeholders to make decisions with them or for them. Government entities, on the other hand, may be able to empower stakeholders and use citizen juries or ballots to determine the final alternative. The same is true for not-for-profit organizations. Whenever a DRB chooses the Consult or Involve levels, the stakeholders should be notified that their input will inform the selection of decision criteria and the weights placed on these criteria. However, it should also be explained that the weights the stakeholders place on the criteria, although important and informative, are nonbinding and in some cases may not directly influence the selection of the final alternative. This is because the weights that the stakeholders place on the various criteria may be different than the weights that members of the DRB might place on the same criteria. In some cases, the highest ranking alternative may be the same when using either the stakeholder’s or the DRB’s weights. In such a case, the DRB and stakeholders will have reached agreement on the same alternative but for different reasons. In many cases, the highest ranking alternative may be different when using the stakeholder’s or DRB’s weights. When this happens, the model results (including sensitivity analysis results (see Chapter 6)) can be used to identify the features of the alternatives that are most contentious. These can be used as a starting point for discussions, negotiations, and even changes to one or more alternatives until an alternative is found that the parties can agree upon or at least accept. Whenever the DRB chooses to have stakeholders involved the collaborate level, the stakeholders should be notified that in addition to informing the selection of decision criteria and weights, they will also assist in the identification of alternatives to be analyzed by the decision model. However, as with the Consult and Involve levels, the DRB will maintain the authority to make the final decision as informed by the stakeholder input, but that they are not bound to a particular alternative based solely on the weights identified by the stakeholders or their preferred alternative.

4.5 Structure Phase The structure phase focuses on those tasks needed to ensure that the decision makers and stakeholders are focused on solving the correct problem and have: – Developed a shared understanding of the issues associated with the decision – Identified their values, objectives, value measures, and preferences – Created a set of creative and implementable alternatives

106

4 The MCDM Process

David C. Skinner notes that the structuring process is often referred to as framing the problem with the goal of all the process participants having a clear and shared understanding of the decision problem [7]. A practice among many decision professionals to assist groups in developing this shared understand is the use of a framing meeting. This framing meeting typically involves a series of facilitated exercises that are designed to not only understand the decision problem but also delay the identification of alternatives until the participants have had the opportunity to think hard about their values, objectives, and preferences prior to identifying alternatives. In other words, to employ value-focused thinking. Before introducing the series of exercises that we have found most useful during the framing meeting, it is important to introduce the following topics: – Concept of the decision hierarchy – The participants in the process and their roles – Preframing meeting activities and exercises

4.5.1 Concept of the Decision Hierarchy The decision hierarchy can be thought of as a conceptual model for establishing a frame and focusing in on the decision(s) that are under consideration. The term frame here is used in a way that is analogous to a photographic picture frame in that it establishes the boundaries for the area the photographer wants to bring into focus by adjusting the camera lens. Figure 4.3 depicts the decision hierarchy as a pyramid with the decision frame in the center. This figure as presented here is from Foundations of Decision Analysis by Ronald A. Howard and Ali E. Abbas [8] and used by permission. The center of this pyramid represents the frame of the decision and includes those strategic decisions (including their associated choice sets) that must be made now. Above the frame are those decisions that are taken as a given and not to be questioned at this time. The top of the pyramid often represents policies, i.e., decisions that have been made regarding how a business or organization seeks to conduct itself. It also includes decisions that have been made about a particular problem, facility, or project. For example, a governmental entity may have decided that a particular portion of land has been zoned for commercial development. Therefore, a corporation seeking to acquire the property would not include in its frame decisions involving development of the property for industrial or residential uses. Below the frame are those decisions can be delayed to a later date or that might become part of the frame for a future decision analysis. Another way to view these lower level decisions is that they represent those that are tactical in nature and include decisions regarding how the currently framed decisions will be implemented. When viewed from top to bottom, another way to think about the decision hierarchy is that the highest level represents the vision, mission, values, or operating

4.5 Structure Phase

107

Taken as Given

To Be Decided Now

To Be Decided Later

Figure 4.3: The decision hierarchy.

policies of a particular business or enterprise. The middle level represents strategic decisions to be made for achieving the vision and accomplishing the mission, in a way consistent with the entity’s values and operating policies. The lowest level in turn represents tactical decisions to be made in deploying the strategic decision (i.e., project management implementation decisions).

4.5.2 The Participants in the Decision Process and Their Roles The selection of the participants of the decision-making process occurs during the structuring phase and prior to conducting the facilitated framing meeting. There are six broad categories of participants. These are presented below and listed in the order that they usually assigned to the decision analysis process. – Decision executive – Decision analysis facilitators – Decision review board – Project team members – Stakeholders – Subject-matter experts 4.5.2.1 Decision Executive The decision executive is the individual that has the highest or final level of approval authority to commit the organizational resources, primarily in the form of money but also in terms of personnel, facilities (e.g., manufacturing operations), and other resources needed to implement the alternative emerging from the MCDM process. In addition, in the case of a business enterprise, the decision executive is often the

108

4 The MCDM Process

individual whose company, division, or department will be most affected by the decision once it’s implemented. In the case of governmental bodies, the decision executive could be a local government leader such as city mayor or leader of a local town council. In some cases, the size and scope of the decision problem may dictate that the decision executive role be shared among several individuals. When this is the case, these individuals will need to establish an agreement regarding how the final decision will be made, e.g., by unanimous agreement, majority vote, or some other allocation scheme. 4.5.2.2 Decision Analysis Facilitators The decision analysis facilitators are responsible for – Ensuring that the MCDM process is followed – Facilitating the framing meeting – Gathering data from project team members and subject-matter experts – Structuring and running the decision model – Producing model results A number of companies operating in industries that were early adopters of decision analysis methods such as the oil and gas and pharmaceutical industries have established internal decision analysis groups, decision analysts, and predefined decision processes. In many ways such companies are at an advantage since the role of the decision analysis facilitator is well understood. In addition, these analysts can be assigned early in the process and even assist the decision executive(s) in selecting the DRB members. A possible disadvantage of such internalized groups is that the process is sometimes simplified and standardized for purposes of ease of implementation and application. As a result, the process may not fit all situations and may be limited to financial decision analysis only rather than the more complex decisions encompassed by MCDM approach. However, the fact that a culture of decision analysis has been created is a great benefit that will lead to more informed decision making and better outcomes. More often than not, a culture of decision analysis has not been established. In such contexts, decision makers facing tough decisions should contract early on with an external decision analyst who can assist establishing and following a quality process, while at the same time providing background training and education on the process while it is being implemented. This includes providing suggestions regarding the selection of DRB members, project team members, subject-matter experts, and level of stakeholder involvement.

4.5 Structure Phase

109

4.5.2.3 Decision Review Board As previously described, the DRB should comprise individuals necessary and sufficient for the decision to be made and successfully implemented. Within a large corporation, the DRB members may consist of senior or high-ranking members from the various company departments. The decision executive(s) may select the members of the DRB or in some cases the DRB is formed and then nominates the decision executive. In either case, the primary role of the DRB is to review the work of the project team, decision analyst, and subject-matter experts and ultimately and in conjunction with the decision executive(s) commit to implement. In addition, to the commitment to implement, the DRB members agree to attend the meetings described in Section 4.3 so that they are informed of the overall process as it progresses. Also they should assist in deciding the level of external stakeholder involvement and may assist in selecting project team members, decision analysis facilitators, and subject-matter experts. 4.5.2.4 Project Team Members The project team members are those individuals that are knowledgeable about the decision situation and have technical knowledge applicable to the problem. Some of the individuals may act as subject-matter experts regarding specific aspects of the decision problem. However, subject-matter experts do not necessarily have to be part of the project team. The project team members are usually from a specific company department such as operations, engineering, finance, legal, health and safety, environmental, and real estate. This group can also include outside contractors and consultants. Depending on the level of stakeholder involvement as decided by the DRB, this project team may also include external stakeholders who are either directly involved or consulted. The project team members are responsible for attending the facilitated framing meeting (and other meetings as necessary), gathering data, performing analysis, and providing data needed for evaluation phase of the process. 4.5.2.5 Stakeholders In Chapter 1, we defined stakeholders as individuals or organizations that are directly or indirectly affected by the outcome of a decision either positively or negatively. We’ve also noted that stakeholders include those who believe that they were affected by the outcome of a decision. Given this definition, it simply would not be possible to include each and every stakeholder in the process (other than beyond the inform level of involvement). However, it is possible to identify stakeholder groups, especially highly influential stakeholder groups (such as Friends of the Green River as described in the case study description), and include individual representatives from such groups into the process. The level of stakeholder involvement should be determined by the DRB.

110

4 The MCDM Process

4.5.2.6 Subject-Matter Experts Subject-matter experts are individuals who have specific knowledge, expertise, and insights in various technical (and possibly even social) elements of the decision problem as well as the relationship between these elements. They can also provide valuable insights into the selection and shaping of probability distributions used to represent random variables within the decision models. The expert elicitation process for the scoring of nonfinancial as well as financial value measures (i.e., costs and revenues) is covered in Chapter 5. Depending on the information and knowledge required, subject-matter experts may be found within any of the six broad categories of participants in the decision analysis process. They can also be individuals who may be working with one of the various divisions of the organization(s) involved in the decision analysis. In addition, they can be individuals who are hired to participate in the process because of their specific knowledge. In some cases, they can be individuals who have access to special knowledge as a result of employment and life experience, who are willing to be interviewed and freely share this information.

4.5.3 Preframing Meeting Activities and Exercises As decision analysts and facilitators, we’ve often been involved with projects involving organizations where a culture of decision analysis and decision quality had not been established. In addition, time constraints associated with the decision would not allow for the type of organization change needed to create a culture of decision quality. In such a situation, the decision executives, members of the DRB, project team members, and subject-matter experts have little knowledge of what to expect from the MCDM process. There are three activities that are extremely valuable for confronting this situation. The first is a preframing meeting including an MCDM overview and process presentation; the second is an online survey; and the third is the prepopulation of an MS Excel template that will be utilized throughout the course of the MCDM process. The purpose of the preframing meeting is to educate to introduce project participants to the complete MCDM process. This includes describing the three phases and nine steps as well as the roles of the various process participants within these phases and steps. In addition, examples of the output of the various steps such as the objectives hierarchy, strategy table, and model results are reviewed. Lastly, a schedule for completing the MDCM process phases as steps is provided. The purpose of the preframing meeting online survey is twofold. The first is to provide background information regarding the decision problem to be addressed. This includes a description of some of the problems or issues to be addressed, strategic decisions that may need to be made, and any risks or uncertainties known at the time. Much of this information will expand and evolve during the MCDM process;

4.5 Structure Phase

111

therefore, the background information provided in the survey is merely an introduction to the issues and complexity of the decision. The second purpose of the survey is to begin gathering information from the project participants regarding their values, objectives, and preferences. The survey includes both free-form questions and structured questions. The freeform questions provide the participants with the opportunity to describe their preferred outcomes or end-state vision and list any issues they see as associated with the decision. “An issue is anything that concerns or influences the possibilities or probabilities of a project – these can be decisions, uncertainties, values, or objectives” [9]. To assist the survey participants with identifying issues, the survey text reminds them that decisions are things under your control, uncertainties are things outside of your control, and values and objectives are things that you want [10]. The structured questions provide the participants with the opportunity to identify value measures (criteria) that they believe are important in evaluating project alternatives. The results of the survey are used for two purposes. The first is to increase the interest and engagement of the framing meeting participants. The survey results are reviewed at the beginning of the framing meeting. This sets the stage for the remainder of the meeting since the participants are now able view the end-state visions, issues, and value measures reported by others and think about them in relation to their own vision, issues, and value measures. An example survey that could be used for the case study example provided in Chapter 2 is provided in Appendix A. The second purpose of the survey is to prepopulate an MS Excel-based MCDM template. During the actual framing meeting a series of exercises are performed to help the project team develop a shared understanding of the issues and a path forward. These exercises are described in the following section. The MCDM template is used to document the results of these activities. In our early days of providing decision analysis services, we would initiate the framing without having conducted the survey or prepopulating the MCDM template. Over time we learned that this slowed down the pace of the meeting and we were unable to complete some of the most important exercises during a one-day framing meeting. In general, this would be acceptable if the participants are willing to attend a two- or three-day framing meeting. However, we’ve found that it can be very difficult to find a date that works for all necessary participants to attend a one-day meeting, let alone one that extends over two or three days. Having portions of the template prepopulated helps accelerate the process because most participants find it easier to comment and provide corrections and additions when a starting point has been established than to begin with a blank slate.

112

4 The MCDM Process

4.5.4 Framing Meeting Exercises Figure 4.4 outlines the various exercises that are performed during the facilitated framing meeting. The purpose of the framing meeting exercises is to provide the decision process participants with a shared understanding of the issues and a vision of a path forward. The completion of these seven exercises marks the completion of the Structuring Phase of the MCDM process. The exercises outlined on Figure 4.4 are performed in order, starting with Background Review at the top of the diagram, i.e., 12 o’clock position, and moving clockwise until Assign Data Gather Tasks, located at the 10 o’clock position. The exercises are designed to be fast-paced and the meeting facilitators work to keep the team engaged, focused, and productive. Two facilitators are recommended for this task with each rotating between a facilitation role and a documentation role, i.e., filling out the MCDM template.

Background Review Assign Data Gathering Tasks

Stakeholder Engagement & Analysis

Shared Vision of Path Forward

Develop Alternatives

Identify Strategic Decisions & Choice Sets

Document Policies

Determine Values, Objectives, & Criteria

Figure 4.4: Framing meeting exercises.

A standardized MS Excel-based MCDM template has been provided for download by the user of this book at https://www.degruyter.com/document/isbn/9783110765861/html. In addition to containing worksheets related to each of the seven framing exercises, the MCDM template also includes additional worksheets for:

4.5 Structure Phase

– – –

113

documenting the meeting attendees, conducting conjoint surveys for the purpose of weighting performance measures (i.e., criteria), and documenting both financial and nonfinancial model inputs

In addition, the MCDM template includes worksheets that contain the MCDM and financial model structure. Lastly the template includes the structures for probabilistic MDCM and probabilistic NPV analysis. The following section discusses the framing meeting exercises and the portions of the MCDM template that are associated with the exercises. 4.5.4.1 Background Information Review The background information review activities help set the stage for the rest of the framing meeting exercises. In the weeks leading up to the framing meeting the participants will have had the chance to attend the preframing meeting presentation (either in person or online) and complete the online survey. Therefore, they should at this point have a general understanding of the MCDM process, a summary of the issues associated with the decision they are facing. In addition, they are often curious to learn the results of the survey. Therefore, the background information review consists of the following activities: – Introduction of the framing meeting participants and their role in the MDCM process – Review of the online survey results – Review of all knowns and unknowns regarding the decision problem A worksheet named “Who is Who” has been provided within the MCDM template for documenting each participant’s – Name – Company or organization they represented – Title – Contact information including email and telephone address – Role – Area of expertise and – If external stakeholder, level of involvement In terms of the overall MCDM process, the role, area of expertise, and the level of involvement of the various participants are their most important attributes. A dropdown menu has been provided within the template for the role to include Decision Executive, DRB Member, Project Team Member, Subject-Matter Expert, and External Stakeholder. The area of expertise field does not include a drop-down menu since

114

4 The MCDM Process

there are too many to predict. The level of involvement field includes a drop-down menu that includes inform, consult, involve, collaborate, and empower. The “Who is Who” template can be prepopulated prior to the framing meeting. However, even when this is done, the attendees should be provided with the opportunity to briefly introduce themselves to the group. The results from the preframing meeting survey should be provided following the introductions of the meeting attendees. In our experience, the attendees are very interested in the results of the survey and enjoy seeing outputs that pertain to values, objectives, and important value measures. It is often discovered during this review that one or more of the survey questions were misunderstood by some of the participants and that they would have answered the questions differently given a better understanding. This is fine since the survey is not the final say on any of these matters but rather a starting point and introduction to the framing process. The framing meeting participants will have the opportunity to correct any such misunderstandings during the framing meeting. The next background activity involves listing knowns and unknowns regarding the decision problem. These knowns, or more appropriately known facts and unknowns (uncertainties or chance events), represent two of the six fundamental elements of decision problems discussed in Section 3.2. It is helpful to place these knowns and unknowns into specific categories. For example, categories that are often helpful with environmental remediation/redevelopment and restoration projects include: – Site history – Surface conditions – Subsurface soil conditions (i.e., soil types, depth to groundwater, type of contaminants present, concentrations of contaminants, horizontal and vertical extent of contamination, and depth to groundwater) – Property status (operating, closed) – Regulatory requirements and issues – Community, media, and public relation issues – Legal issues – Health and safety issues A worksheet titled “Knowns and Unknowns” has been included in the MDCM template. The category names included in this template have been left generic and are simply labeled Category One, Category Two with a total of ten categories included. Each category is divided into two halves, i.e., knowns and unknowns. Prior to the framing meeting, many of the categories can be named and prepopulated using the results of the unstructured survey questions. As part of the background review activities, the facilitators will work with the attendees to refine the category names and expand on the list of knowns and unknowns associated with each category.

4.5 Structure Phase

115

4.5.4.2 Stakeholder Analysis and Engagement Stakeholder analysis and engagement for the purposes of MCDM should go beyond the traditional stakeholder management process typically applied to large capital or technical projects. Therefore, it is useful to review the traditional process before discussing ways of going beyond that process. The need for stakeholder management has long been recognized by those involved with the management of large-scale capital and technical projects. In writing about strategic project management David Cleland states [11]: Successful project management can be carried out only when the responsible managers take into account the potential influence of the project stakeholders. An important part of the project planning is the identification of all project stakeholders and their relevant stake in the project. Stakeholder analysis during the planning of the project is particularly useful for the development of strategies to facilitate the “management” of the stakeholders of during the life cycle of the project.

Cleland goes on to state that “failure to recognize or cooperate with adverse stakeholders may well hinder a successful project outcome. Indeed, strong and vociferous adverse stakeholders can force their particular interest on the project manager at any time, perhaps at the time least convenient to the project” [12]. These statements certainly make a strong case for stakeholder management. However, the focus of the process suggested by Cleland is on the “management” of stakeholders in a way that prevents them from having a negative impact on the project’s outcomes. It is more about management control rather than engagement. However, the steps suggested by Cleland can be used as a foundation for stakeholder engagement. For example, Cleland suggests that stakeholders who may attempt to exert an influential control on a project should be analyzed and cataloged. He suggests that the following issues should be addressed: – Who are the most formidable stakeholders? – What are their strengths and weaknesses? – What is their strategy and the probability of their being able to implement such as strategy? – Do any of these factors give the stakeholder a distinctly favorable position which can influence the project outcome? The Project Management Institute, in its Sixth Edition of A Guide to the Project Management Body of Knowledge (PMBOK Guide), suggests categorizing stakeholders with respect to a number of dimensions including [13]: – Internal/external – Level of authority (power) – Level of concern about the project’s outcomes (interest) – Ability to influence outcomes (influence)

116

4 The MCDM Process

The Sixth Edition of the PMBOK Guide notes that, at the time of its writing (2017), new trends and practices were emerging that go beyond stakeholder analysis and management to include stakeholder engagement. The trends included broader definitions of stakeholders that go beyond the traditional categories of employees, suppliers, and shareholders to include groups such as – Regulators – Lobby groups – Environmentalists – Financial organizations – The media – Those that believe they are stakeholders (i.e., they believe they will be affected by the project) [14] These emerging practices include but are not limited to – identifying all stakeholders, not just a limited set; – consulting with stakeholders most affected by the work or outcomes through the concept of co-creation; and – capturing the value of stakeholder engagement both positive and negative [15]. The concept of co-creation simply means including affected stakeholders in the project team as partners. Regarding the MCDM process, co-creation would be consistent with choosing one of the higher levels of stakeholder involvement such as involve, collaborate, or empower. Regarding capturing the value of stakeholder engagement, an example of positive value would be the benefits gained as a result of active support provided by local politicians, community leaders, and business executives. An example of capturing the negative value would be measuring the costs of poor stakeholder engagement such as project delays, project cancellation, and loss of reputation. The MCDM template includes a worksheet named “Stakeholder Summary.” This worksheet is designed to go beyond traditional stakeholder analysis and includes data fields that are consistent with stakeholder engagement. The template includes the following data fields: – Stakeholder Name; – Issue/Stake; – Level of Authority (Power); – Level of Concern (Interest); – Ability to Influence Outcomes (Influence); – Priority; – Level of Stakeholder Involvement; – Management Strategy

4.5 Structure Phase

117

The stakeholder’s Name and Type fields are self-explanatory. The Issue/Stake field is used to describe the stakeholder’s actual or perceived stake or issue. Each of the next three fields – Power, Interest, Influence – have drop-down menus with a scale of 1 to 5 (lowest to highest for each of those factors or attributes). Stakeholder priority thus can be calculated numerically, with 5 representing the highest priority. The Level of Stakeholder Involvement field includes a drop-down menu with the words Inform, Consult, Involve, Collaborate, and Empower. The framing meeting attendees may choose to relate these levels to the priority scale. However, a one-to-one match is not necessary since, as previously discussed, in many cases it would not be possible to empower external stakeholders. 4.5.4.3 Document Policies As previously defined in Section 4.5.1, policies represent items taken as given. These include decisions that have been made regarding how a business or organization seeks to conduct itself or regarding the decision problem under investigation. Policies should include any laws, regulations, or other requirements that apply directly to the decision problem. It might seem that this exercise would be relatively easy to complete. However, during many framing meeting sessions we’ve often learned that there are disagreements regarding whether certain decisions are indeed policy decisions. This can and does lead to active and energetic discussions. This is not a negative situation. Rather, it is one of the steps that the group takes toward a shared understanding of the issues and a shared vision of a path forward. 4.5.4.4 Develop Objectives Hierarchy Developing the objectives hierarchy is often one of the most difficult tasks of the entire MCDM process. It requires not only thinking hard about values, objectives, and value measures but also organizing them into a logical hierarchical structure. There are several approaches that can be used to help facilitate this process; i.e., top-down, bottom-up, and blended approach. 4.5.4.4.1 Top-Down Objectives Hierarchy Approach To perform the top-down approach, the framing meeting attendees, with the help of the decision analysis facilitators, work to organize values and objectives in top-down structure with the most important of highest level values/objectives placed at the top (i.e., fundamental objectives) of the hierarchy and the lower level objectives (i.e., means objectives) placed below and feeding into the higher level objectives. There is no limit to the number of levels of means objectives. However, two to three levels are usually sufficient, and in some cases, one level is sufficient. The process continues until a singular value measure (i.e., criteria or evaluation measure) can be associated

118

4 The MCDM Process

with each of the lowest level objectives. As defined in Section 1.5.12, value measures are scales that indicate the degree of attainment of an objective. The top-down approach can be difficult and slow especially if the process begins from scratch. To avoid this problem, the answers that the attendees provide to preframing meeting “issues” questions in the premeeting survey can be used to accelerate this process. This is done by having the facilitators use the survey results to begin developing a top-down objective hierarchy. This predeveloped hierarchy is then presented at the beginning of the exercise. The MCDM template includes a worksheet named “Objectives Hierarchy” that can be used to both begin the objectives hierarchy process prior to the meeting and to continue the process during the meeting. At the start of the exercise, it is not necessary for the facilitators to include every objective in the predeveloped hierarchy. However, it is necessary that all objectives identified by the survey be reviewed as part of this exercise. The facilitators should not discard or remove any of the identified objectives unless agreed upon by the framing meeting attendees. Oftentimes, there are several objectives identified during the preframing meeting survey that are worded differently but mean the same thing. During the objectives hierarchy exercise these can be blended into the one objective. In other cases, new objectives are identified as the framing meeting attendees work to construct the objectives hierarchy. These are incorporated into the objectives hierarchy assuming that the attendees agree on the addition. 4.5.4.4.2 Bottom-Up Objectives Hierarchy Approach Many individuals have found that it is often easier for them to identify value measures than it is to articulate values and objectives. For example, they may feel it’s important to measure greenhouse gas (GHG) emissions or the number of full-time equivalent jobs created. Of course, these are value measures that point toward specific values and objectives such as minimizing the contribution to climate change or creating a strong economy. Rather than forcing such individuals to try to work in a top-down fashion until they eventually reach these value measures, the facilitators simply ask such individuals questions such as: – Why is this measure important to you? – What objective do you think is served by minimizing or maximizing this value measure? – Does the objective you’ve mentioned in relation to this value measure serve an even higher level objective or overall value? Continuing in this manner it is possible to work upward from the value measures to complete the entire hierarchy. It should be noted that a list of the values measures will be available as a result of the preframing meeting survey.

4.5 Structure Phase

119

4.5.4.4.3 Blended Objectives Hierarchy Approach The blended objectives hierarchy approach, as the name indicates, simply means working simultaneously in a top-down and bottom-up fashion until the objectives hierarchy is completed. As one might expect, this often the most effective way to complete the hierarchy, the one that is used most often in actual practice, and the one that is most efficient. This is because during the objective hierarchy exercise the facilitators can transition between approaches at any point in the process where the meeting attendees have paused or are having trouble identifying next or lower level objectives or, conversely, higher level objectives. 4.5.4.5 Example Objectives Hierarchy Figure 4.5 presents an example of an objectives hierarchy that was developed for a remediation/solar redevelopment project. As such, this hierarchy includes objectives and value measures consistent with the US EPA’s CERCLA process as well as sustainable redevelopment/sustainability analysis. The highest level and most fundamental objective as seen at this top of this diagram is a Clean Environment & Sustainable Redevelopment. Beneath this fundamental objective we see that there are four means objectives including positive: – Ecological/Environmental Impacts – Community and Economic Impacts – Financial/Regulatory Impacts – Owner Impacts Note that these means objectives could have been presented all on the same level in a tree-like diagram. However, they are presented adjacent and beneath each other in this diagram to make the figure more compact. The values measures associated with each of the four lower level “means objectives” are shown beneath them in an expanded format. Note that there can be more than one value measure associated with each means objective; this is often the case. In this example we see four value measures associated with Community and Economic Impacts, Owner Impacts, and Financial Regulatory impacts. The fourth means objective, Ecological and Environmental Impacts has a total of five value measures. The units of measure for each of the value measures are presented directly beneath their name. Whenever possible it is best to use value measures that can be measured in natural units such as kilowatt-hour (kWh), acres, or years. However, this is not always possible and numerical scales must be created to define the value measure, such as a scale of 1–10. Whenever this is done, descriptive text that explains the attributes or situation that would be represented by the numerical values within the scale must be provided. It is important to note that the value measures have a directionality associated with them meaning that for some of the value measures a larger numerical value is

Jobs FTEs

Remediation Cost PV $

Solar Power Development NPV $

Contribution to Renewable Power Standards % Remediation Complete Calendar Year

Vegetative Cover Impacts Acres

Solar Feasibility Knowledge Gained Scale 1 to 10

Sensitive Species Affected #

Time until Groundwater Preservation of Returns to Baseline Greenfields Conditions Acres Years

Lifecycle GHG Emissions Tons CO2

Power Transferable Contract Implementation Negotiation Process Leverage Scale 1 to 10 Scale 1 to 10

Positive Site Owner Impacts

Positive Ecological/Environmental Impacts

Regulatory Agency Relationship Scale 1 - 10

Wt = 25%

Wt = 12.5%

Community Perception Scale 1 to 10

Positive Site Owner Impacts Wt = 25% Positive Ecological/Environmental

Clean Environment & Sustainable Redevelopment

Positive Financial/Regulatory Impacts

Green Energy Local Economic for Impact Community $ kWh

Figure 4.5: Example objectives hierarchy.

Criteria

Positive Community & Economic Impacts

Criteria

Positive Community & Economic Impacts Wt = 37.5% Positive Financial/Regulatory

Criteria Criteria

120 4 The MCDM Process

4.5 Structure Phase

121

better and for others, a lower numerical value is better. For example, when it comes to Green Energy for the Community measured in kWh a larger numerical value or score is desired. However, for a value measure such as Life Cycle GHG Emission measured in tonnes, a lower score is desired. 4.5.4.6 Identifying Value Measures (Criteria) The process of identifying value measures can begin with the preframing meeting survey and continue up to the time that the objectives hierarchy is complete. The importance of the value measures (i.e., criteria) cannot be overstated since, as representative measures of the decision makers’/stakeholders’ values and objectives, they are the primary drivers for the ranking of alternatives. Therefore, questions often asked by framing meeting attendees include: – What are the characteristics of a high-quality set of criteria? – Is there a way to identify a good starting point or preliminary set of criteria? – Is there a limit to how many criteria can be used in the analysis? The characteristics of a high-quality set of criteria are that they: – Avoid double counting – Are conceptually independent – Are stated in natural units, whenever possible and – Include textual descriptions whenever they must be defined categorically or in terms of numerical scales 4.5.4.6.1 Double Counting Whenever anything is being summed, it understood that double counting is to be avoided. However, unless careful inspection and consideration is applied, it is possible to establish criteria that are indeed counting the same thing. For example, a criterion such as acres of habitat may be established of measuring the objective of protecting the environment. Another criterion such as acres of greenspace might be established as way to measuring community impacts (i.e., preventing encroachment into natural areas). Upon further inspection, the group may realize that these two criteria are measuring the same thing and choose to include one or the other. 4.5.4.6.2 Conceptual Independence Conceptual independence is more complex than mere double counting and since at first blush it can seem as if we are now allowing for double counting. The basic idea is that criteria should be conceptually distinct. Benjamin F. Hobbs and Peter Meir note that a strict type of conceptual independence is called preference independence [16]. Hobbs and Meir further go on to note that:

122

4 The MCDM Process

Decision analysis differentiates between statistical independence and preferential independence; the former refers to a correlation structure of the alternatives and, the latter to a structure of the user’s preferences – a distinction that might be characterized as “facts” versus “values” [18].

An example of preferential independence from a client engagement involved two different financial parameters: present value cost and total escalated cash flow. These two parameters are calculated based on the same underlying series of nominal cash flows. Therefore, they are indeed strongly correlated. However, this client had made many decisions regarding future management of environmental cleanups based primarily on present value. In the years since the decisions were made, they discovered that not only had they underestimated the future cost of cleanup, but the size of the cleanup they were faced with had grown considerably. Therefore, they did not want to use net present value cost as their sole financial metric. They also chose to include total escalated cash flow as a metric. With this approach, they still consider prevent value cost but they placed more weight on escalated total cash flow. 4.5.4.6.3 Stated in Natural Units Stating criteria in their natural units of measure is a best practice since it allows decision makers and perhaps most importantly external stakeholders visualize the total consequences associated with various alternative. 4.5.4.6.4 Categorical Criteria There are cases where natural units across numerous criteria vary so widely that it is difficult for stakeholders to make trade-offs and therefore category values are a useful addition. This is done by developing clear descriptions that define what is meant by phrases like mild, severe, opposed, or approved. Numerical scales such as one to five are then applied to the terms and their descriptors. 4.5.4.6.5 Identifying a Starting List of Criteria In general, it would be best to allow the criteria to evolve naturally and as an outgrowth of thinking hard about values and objectives. However, in most cases, this can be a time-consuming and laborious process. Therefore, having a starting list of possible value measures can help accelerate the process. The question then becomes where does one look at a starting point for such criteria. For those interested in improving their organization’s ESG performance, a review of the United Nations Sustainable Development Goals can be very helpful. Another approach is to think in terms of the three pillars of sustainability – i.e., social, economic, and environmental – to identify criteria that could be used to indicate an improvement in any of these areas. For those involved in the environmental remediation industry, the following reference documents can be very helpful:

4.5 Structure Phase

– – –

123

ASTM E2893 Standard Guide for Greener Cleanups. Wes Conshohocken: ASTM International Holland, K. S., Lewis, R. E., Tipton, K., et al., Framework for integrating sustainability into remediation projects. Remediation Journal, 2011, 7–38. U.S. Environmental Protection Agency. (2012). Methodology for Understanding and Reducing a Project’s Environmental Footprint. U.S. Environmental Protection Agency.

Following is a potential starting list of criteria that we’ve seen frequently used in the remediation/restoration industry. However, they are general in nature and may apply to a wide variety of industries and companies. Social criteria – Number of full-time equivalent jobs – number FTEs – Local economic impact – $ millions – Recreation areas added – number new soccer fields, baseball fields, basketball courts, etc. – New housing units – number – Road repair/improvement – miles – Diversity – categorical scale – Green energy – kWh – Community perception – categorical scale Environmental criteria – GHG emissions – tons CO2 – Sensitive species affected – number and type – Preservation of Greenfields – acres – Achievement of statewide maximum contaminant levels for contaminants of concern in soil or groundwater – Wetlands restoration – acres Economic criteria – Net present value – $ millions – Capital expenditures – $ millions – Annual operating expenses – $ millions 4.5.4.7 Designing Alternatives The final step in the structuring process is the creation of a strategy table. In the terminology section of Chapter 1.0, an alternative was defined as a collection of strategic choices. Furthermore, it was noted that during the planning stage of nearly every technical project that it is seldom the case that there is only one strategic decision to be made. Rather there are many strategic decisions to be made and as discussed in Section 3.2.1 each strategic decision contains its own finite choice set. Strategy tables

124

4 The MCDM Process

are powerful communication tools that allow decision makers and stakeholders to visualize the set of strategic choices that make up each alternative. The process of creating the strategy table begins by placing the name of each identified strategy decision as a column heading within the table. The available choices associated with the strategic decision are then listed beneath the column heading. It is important to make sure that the strategic decisions listed in the table are at the proper level of focus. Referring to decision hierarchy (Figure 4.3), the decisions included in this table are not policy decisions (which are taken as a given) nor are they tactical decisions that pertain to implementation that can be deferred to the future. Rather they are evaluation decisions that are to be decided upon now and are the focus of the framing session. Once the strategy decisions and their choice sets have been included in the table, a new leftmost column is added to the table and given the heading Alternative Theme. The process of creating alternatives then begins with identifying alternative themes or descriptors that define what the alternative is intended to achieve. Examples of possible themes as they relate to our case study might include Business Friendly, Industrial Development, Mixed Community, Ecologically Friendly, or Balanced Development. Once the themes have been named, the framing meeting participants then work to identify choices from each column that are consistent with the theme. Figure 4.6 displays a conceptual strategy table that includes two alternatives with their associated strategic choices.

Alternative Theme

Decision 1

Decision 2

Decision 3

Decision 4

Figure 4.6: Example strategy table.

To many that strategy table may seem rather simplistic. However, it is a powerful and effective tool. Ron Howard, who is widely recognized as one of the founders of decision analysis and applied decision theory, has stated: The most important idea in creating alternatives that I have encountered is the strategy generation table . . . When I first came across the strategy table, it seemed rather simplistic to me from a technical point of view. I had criticism, such as “We are not doing a complete set of alternatives.” Yet I found that there were few ideas in decision analysis that responded to the multiplicity of possible strategies in the strategy problem. As a result, I came to regard the strategy-

References

125

generation table not as a quick and dirty approach, but rather as a very useful tool for helping people think their way through problems where there were literally thousands of possible strategies [18].

We could not agree more with Dr. Howard’s assessment of the strategy table. We have found it extremely useful in framing sessions to help identify alternatives that the attendees felt were valuable, implementable, and provide a sufficient spectrum of themes that could be pursued. However, we should point out that using Lumivero’s RiskOptimizer, it is possible to set up a genetic algorithm optimization that will mix and match choices (as long as proper constraints are applied) that will evolve to solutions that the group might not have otherwise identified. We have used this approach on two projects. However, explanation of how to do this is beyond the scope of this book.

4.6 Exercises This section uses the case study information to develop the following items: – Stakeholder analysis – Objectives hierarchy diagram – Strategy table A blank MCDM template has been provided that can be used to complete these exercises. This template can be accessed at https://www.degruyter.com/document/isbn/ 9783110765861/html. We recommend that these exercises be completed from the perspective of the mayor and city council. A completed stakeholder analysis, objectives hierarchy diagram, and strategy table based on the Chapter 2 case study can be found in Appendices B through D, respectively.

References [1] [2] [3] [4] [5] [6] [7]

Skinner, D. C. Introduction to decision analysis – A practitioners guide to improving decision quality, Second Edition, Gainesville, FL, USA, Probabilistic Publishing, 1999. Hobbs B. F., Meier, P. Energy Decisions and the Environment. New York, NY, USA, Springer Science and Business Media, 2000. Parnell, G. S., Bresnick, T. A., Tani, S. N., & Johnson, E. R., Handbook of Decision Analysis, John Wiley & Sons, Inc. Hoboken, NJ, USA, 2013. Keeney, R., Making better decision makers, Decision Analysis, 2004, 1(4),193–204. doi:10.1287. Parnell, G. S., Bresnick, T. A., Tani, S. N., & Johnson, E. R., Handbook of Decision Analysis, John Wiley & Sons, Inc. Hoboken, NJ, USA, 2013, pp. 97. Inviting Stakeholders to decision making, Organization Development, (Accessed August 14, 2022 at betterboards.net:https://betterboards.net/org-dev/inviting-stakeholders-decisionmaking). Skinner, D. C. Introduction to decision analysis – A practitioners guide to improving decision quality, Second Edition, Gainesville, FL, USA, Probabilistic Publishing, 1999, pp. 124.

126

[8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18]

4 The MCDM Process

Howard, R. A., & Abbas, A. E., Foundations of decision analysis, Upper Saddle River, New Jersey, USA, Pearson Education, Inc., 2016, pp. 340. Skinner, D. C. Introduction to decision analysis – A practitioners guide to improving decision quality, Second Edition, Gainesville, FL, USA, Probabilistic Publishing, 1999, pp. 127. Ibid. pp. 128. Cleland, D., Project Management, Strategic Design and Implementation, USA, New York, NY, McGraw-Hill, 1990, pp. 98. Ibid. pp 103. Project Management Institute, A Guide to the Project Management Body of Knowledge, USA, Newtown Square, PA, Project Management Institute, Inc., 2017, pp. 512. Ibid. pp. 505. Ibid. Hobbs B. F., Meier, P. Energy Decisions and the Environment. New York, NY, USA, Springer Science and Business Media, 2000, pp. 22. Ibid. Howard, R. A., Decision analysis: Practice and promise, Management Science, 1988, 34(6),679–695. (Accessed from https://doi.org/10.1287/mnsc.34.6.679).

5 The Evaluation Process – Building the MCDM Model The completion of the framing exercises marks the end of the structure phase of the MCDM process and the beginning of the evaluation phase. Although all the faming exercises are important the most important outcomes for entering the evaluation phase are the – objectives hierarchy; – strategy table; and – assignment of the data gathering tasks. The objectives hierarchy is important because it structurally links values, objectives, and value measures. The strategy table is important because it identifies the alternatives to be evaluated. Last, the assignment of data gathering tasks means that individuals (or groups) have been identified who have the expertise necessary for providing model inputs. This chapter focuses on three steps that make up the evaluation phase: – Quantifying preferences and uncertainties – Structuring the model – Performing probabilistic analysis (Monte Carlo simulation)

5.1 Quantifying Preferences and Uncertainties Quantifying preferences is the process of assigning weights to value measures identified during the framing meeting and presented at the lowest level of the objectives hierarchy. Preferences are determined by the willingness of individuals or groups to make trade-offs. In Chapter 1, we noted that trade-offs involve giving up a little of something valued to gain more of something that is valued even more. In this chapter we review the use of conjoint surveys as an effective way of helping decision makers/ stakeholders articulate their willingness to make subjective trade-offs. In addition, we describe a process for objectively analyzing these trade-offs to determine the decision-maker/stakeholder preferences in the form of criteria weights. There are two primary types of uncertainties that must be quantified. These include: – the range of values that the various model input parameters can assume; and – the probabilities associated with chance events that may or may not occur. Quantifying uncertainties regarding the range of values that the various criteria and other model input parameters may take on involves assigning theoretical PDFs. Assigning PDFs to uncertain input parameters can be based on actual data or expert judgment. If actual data is available, then the best and most representative way of https://doi.org/10.1515/9783110765861-005

128

5 The Evaluation Process – Building the MCDM Model

quantifying uncertainties is to fit a PDF to the actual data. This can be done using the distribution fitting feature of @Risk. This feature was used to produce Figure 3.13 (see Chapter 3) which was created by fitting the Weibull distribution to Phase B remediation investigation data. In Section 5.3, we review the use of @Risk’s distribution fitting feature and describe the process used to select the Weibull distribution to represent uncertainties associated Phase B investigation costs. It is often that case that little or no data exists for most of the required input parameters. This is because the majority of MCDM models tend to be issue and projectspecific. Therefore, at some point in the past, no one had the foresight to establish a database and begin collecting data for the parameter in question. When little or no data exists, expert judgment must be used to assign PDFs to the input parameters in question. In Section 5.4, we describe the expert elicitation process and methods for calibrating experts so that they can provide information that adequately captures the range of uncertainties associated with the input parameters that they are estimating. Chance events that may or may not occur fall into two broad categories: naturally occurring and human-induced. Examples of naturally occurring chance events include hurricanes, tornados, earthquakes, and floods. Examples of human-induced chance events include lawsuits, regulatory changes, protests, and boycotts. In some cases, it can be difficult to categorize chance events as naturally occurring or humaninduced. For example, an unusually powerful hurricane could simply be the result of natural trends or of human-induced climate change. Similarly, the mechanical failure of a piece of equipment may be the result of normal wear and tear, poor engineering design, or improper use. Fortunately, assigning categories to such events is not as important as assigning the probability of occurrence. In some cases, probabilities can be assigned based on extensive data regarding the frequency of such events (i.e., using a frequentist’s approach). In other cases, the probabilities will be subjective in nature and must be estimated based on the weight of evidence (i.e., using Bayesian methods).

5.2 Conjoint Surveys Throughout this book we’ve noted that the weights assigned to various value measures are based on the subjective preferences of the decision makers and stakeholders involved in the MCDM process. Furthermore, we’ve stressed that the willingness of individuals to make certain trade-offs provides the information needed for understanding their preferences. Last, we’ve noted that since the willingness to make trade-offs is based on subjectivity, such willingness can be very hard for individuals and groups to articulate without the aid of a structured process for drawing out this information. There are many methods available for weighting criteria including simplistic equal values, observed derived weights using linear regression techniques, direct weighting, the analytical hierarchy process, swing weighting, indifference trade-off weights, and conjoint surveys [1]. Benjamin F. Hobbs and Peter Meir provide a good

5.2 Conjoint Surveys

129

summary of all these methods, except for conjoint surveys, in Energy Decision and the Environment: A Guide to Multicriteria Methods [2]. Rather than describing the process of implementing all these various methods, along with their pros and cons, we focus on conjoint surveys analyzed with the aid of linear regression techniques as our preferred method. Conjoint surveys are preferred because they require decision makers and stakeholders to carefully consider their priorities and make trade-offs. When scoring alternatives, they must compare alternatives that have different levels of performance across the value measures. If Alternative A creates more jobs than Alternative B, but Alternative B protects more habitat than Alternative A, the respondent will need to think through which alternative is better and why. And, by discussing their choices with other stakeholders, it can increase consensus about the meaning and importance of each attribute. By using a decision context, conjoint models mimic the way individuals make decisions in a real-world setting such as when deciding on a new automobile, house in a new neighborhood, or choosing which job offer to accept. In addition, the weights that result from the conjoint survey/linear regression method that we describe here are objective in that they are statistically derived, yet subjective in that they are based on trade-offs. Lastly, the conjoint survey approach is well-suited to use within MS Excel, where the rest of the MCDM model will reside.

5.2.1 Administering the Conjoint Survey Conjoint surveys can be administered in-person or by online surveys. In-person meetings led by experienced decision analysis facilitators will typically provide the most reliable data. The benefit of the in-person meetings is that the facilitators can assist the group in maintaining focus and momentum throughout the process as the group struggles to make difficult trade-offs. Most find the conjoint survey easy to understand in that they are simply required to score a set of alternative scenarios. However, there are often initial disagreements or confusion about the definition of the attributes and whether they are “realistic.” For example, an attribute might be acres of restored habitat, but the conversation might quickly show there are different interpretations of what “restored” means (e.g., pristine vs. conditions like other nearby sites) or how quickly the land will be restored. Or some respondents can get caught up trying to make the outcomes for a specific alternative “realistic” (e.g., How can Alternative A produce 1,000 jobs and 200 restored acres, while Alternative B produce 500 jobs, but only increase restored acres to 220?). Respondents often need to be reminded that the conjoint survey is about what people want, and it has nothing to do with what is technically feasible. Technical feasibility is a separate process and, at the end of the MCDM process, what people want needs to be merged with what people can get. Finally, the discussions and clarifications can also lead to consensus.

130

5 The Evaluation Process – Building the MCDM Model

When administering the surveys using in-person meetings, it may be worthwhile to use separate breakout groups. They can be broken down by decision makers, stakeholders, or by separate stakeholder groups representing different interests. This is because one goal of the MCDM process as applied to a given project may be to obtain weights representative of differing stakeholder groups. These weights from the various groups can be used to run the MCDM model to crystallize the alternatives that are most acceptable and the outputs that are generating the most consensus or disagreement among groups. One important benefit of MCDM and in-person meetings is that it can efficiently identify some areas of consensus and provide some early successes. Administering the conjoint surveys online is efficient in that individuals are able to take the survey at their convenience. In addition, online surveys can be designed to collect demographic and other important metadata that can help understand the preferences of various stakeholder groups. This enables the analysts to establish average alternative scores from differing stakeholder groups. However, when the online process is used, there is a risk that the participants will rush through the survey and assign scores without giving the trade-offs adequate consideration and without the benefit of the dialogue and discussion that occurs during in-person meetings. In general, regardless of which approach is used, it is important to remind all stakeholders that the scores are used to GUIDE decision makers, not MAKE decisions. So there is no need to “weight” the values to be reflective of the entire community or affected stakeholders. Scores are used to understand and explore preferences in a systematic, reliable manner to facilitate decision making.

5.2.2 Example Conjoint Survey Tables 5.1 and 5.2 present the conjoint survey used to derive the weights for the value measures associated objectives hierarchy in Figure 4.5. This example focuses on the positive community and economic impacts portion of the hierarchy. To complete the weights for the entire hierarchy, the decision makers/stakeholders are required to complete a total of five conjoint surveys, one for each of the four means objectives contained in the hierarchy and one that compares the four means objectives against each other. Tables 5.1 and 5.2 are screenshots of portions of the survey as it exists within an MS Excel worksheet. Table 5.1 is referred to the criteria definitions portion of the survey because its purpose is to – name the value measures (i.e., criteria); – indicate their units of measure; and – provide descriptions of what would constitute a really good outcome and not so good outcome for each measure. In describing what constitutes a really good and not so good outcome for each criterion, the survey designers (usually the decision analysis facilitators) should seek to

5.2 Conjoint Surveys

131

establish outcomes representative of the range across all the alternatives identified during the framing session. This is not always possible since at this point in the process, the subject-matter experts (SMEs) may not have scored the alternatives against the value measures. Therefore, the survey designers, in consultation with SMEs, when possible, will need to establish outcome ranges for each criterion that they believe is reflective the potential range across all alternatives. It is not necessary that these ranges be perfectly accurate within the survey since they are used simply to assist the survey participants in making trade-offs. Table 5.1: Example conjoint survey criteria definitions. Annual jobs

Annual green energy to local community

Local economic impact

Community perception

 = Greater than  FTEs = Zero FTEs

 =  mWh  =  mWh

 = $ million  = $ million

 = Stakeholders favor redevelopment plans  = Lawsuits filed or significant negative stakeholder reaction

Really good outcome









Not so good









Description

Note that numerical values of 1 and 5 have been assigned as representative of “not so good” and “really good” outcomes rather than using the units in the value measure descriptions. This is done to ensure that meaningful, representative, and statistically significant weights can be provided by the linear regression used to analyze the survey. These representative values (i.e., scale of 1–5) are not the values used in the objective function that will be used to analyze alternatives. The MCDM objective function, presented in Section 5.6.1, makes use of the actual scores for each value measure reported in their natural or proxy units. Therefore, when scoring the alternatives, the scores will be provided in the units included in the description for each value measure. Table 5.2 presents the alternatives scoring portion of the survey. At the beginning of the survey, all cells within this table are prepopulated, except for those contained within the far-right hand column, i.e., the score column. During the survey, the participants are asked to score the alternatives on a scale of 1–5, 1 being the least favorable and 5 the most favorable. In the example, the scores have already been assigned. Note, it is not necessary that that integer values be utilized when scoring the alternatives, decimal values are permitted.

132

5 The Evaluation Process – Building the MCDM Model

Table 5.2: Example conjoint survey alternatives scoring table. Outcome scenarios

Scoring definitions  = Highest possible score  = Lowest possible score

Number

Annual jobs

Annual green energy to local community

Local economic impact

Community perception

Score











.











.











.











.











.











.











.











.

The number of alternatives as well as the criteria states or levels within each of the alternatives is based on design of experiments approach. Details regarding the design of experiments approach are provided in the following section.

5.2.3 Design of Experiments The design of experiments approach makes it possible to tease out value measure weights while minimizing the number of alternatives that decision makers/stakeholders must compare. In other words, it enables the most efficient survey design. Our conjoint surveys make use of Taguchi orthogonal arrays using the procedures described by Raghu N. Kacker, Eric S. Lagergren, and James J. Fillibren in their paper Taguchi’s Orthogonal Arrays Are Classical Designs of Experiments published in the Journal of Research of National Institute of Standards and Technology in 1991 [3]. According to Kacker, Lagergren, and Fillibren “Orthogonal arrays can be viewed as multi-factor experiments where the columns correspond to the factors, the entries in the columns correspond to the test levels, and the rows correspond as the test runs” [4]. For our use in conjoint surveys, the columns correspond to impact/value measures (i.e., criteria), the entries in the columns correspond to value measure levels and the rows refer to alternatives or scenarios. Note that Table 5.2 contains a total of eight scenarios. This experimental design is a subset of a complete factorial design. A complete factorial design is one where the alternatives would represent all possible combinations of criteria levels. The number

5.2 Conjoint Surveys

133

of alternatives needed for a complete factorial design can be calculated using equation (5.1). In this equation “A” represents the number of alternatives (or test runs in design of experiments terminology), “m” represents the number of criteria (i.e., factors in design of experiments terminology), and “s” represents the number test levels associated with the criteria. A = sm

(5:1)

Therefore, a full-factorial design for the 4 criteria contained in Tables 5.1 and 5.2 and their associated two levels would require a total of 16 alternatives since 24 = 16. The use of the Taguchi orthogonal arrays allows for a design involving only 8 alternatives instead of 16. In their paper, Kacker, Lagergren, and Fillibren denote orthogonal arrays using the symbolic notation OAN ðsm Þ, where OA stands for orthogonal arrays, N is the number of rows (i.e., test runs or alternatives), m is the number of columns (i.e., factors or criteria), and s is the number of test levels for each factor. Using this notation, the   orthogonal array presented in Table 5.2 would be denoted as OA8 24 . The authors note that then N rows of an OAN ðsm Þ can be viewed as an N=sm fraction of a complete sm factorial design. Therefore, Table 5.2 can be viewed as a 8=24 =1=2. fraction of a complete 24 factorial plan. Now that we have defined the terms for the elements that make up orthogonal arrays (i.e., s, m, and N) and the method of denoting orthogonal arrays; we can provide a formal definition of such an array. An orthogonal array denoted by OAN ðsm Þ is an N × m matrix whose columns have the property that in every pair of columns each possible ordered pairs of element appears the same number of times [5]. This is seen in Table 5.2, where for every pair of columns each of the four ordered pairs (1,1), (1,5), (5,1), and (5,5) appears exactly two times. Kacker, Lagergren, and Fillibren provide the mathematics necessary to create a wide variety of Taguchi orthogonal arrays. The MCDM template provided along with this book contains conjoint survey designs for: – Three criteria by two levels (i.e., three by two) – Four by two – Five by two – Six by two – Seven by two – Four by three – Five by three For most projects, the first three in this list (i.e., three by two; four by two; and five by two) are all that is needed. This is especially true if the objectives hierarchy contains two or more levels such as presented in Figure 4.5.

134

5 The Evaluation Process – Building the MCDM Model

5.2.4 Evaluating Conjoint Surveys Using Linear Regression The weights for the individual criteria are calculated using a multivariate linear regression model (hereafter linear regression) in which the outcome variable is related to multiple explanatory variables. The following equation provides the general equation for a linear regression model: yi = b0 + b1 x1i + b2 x2i +    + bk xki + εi

(5:2)

The symbols b0, b1, b2, . . ., bk represent the coefficients of the regression model and x1i, x2i, . . ., xki represent numerical values of the explanatory variables. The symbol on the left-hand side of the equation, yi represents the dependent variable. Last, the symbol εi represents the residual or error term. The coefficients and the error term as presented in equation (5.2) are generated using a linear regression. The bk terms represent the expected change in the outcome variable yi (e.g., the score) given a change in their associated explanatory variable xk (e.g., the impact criteria) when holding all other explanatory variables constant. More technically, bk is the partial derivative (slope or rate of change) of the expected outcome yi given a change in xk . Therefore, the relative magnitude of each coefficient, with respect to the other coefficients, is an indication of their contribution to the value the dependent variable. The linear regression approach based on minimizing the sum of the squared residuals (SSE), that is, the squared error term. The following equation presents the formula for calculating the SSE: SSEðbo , b1 , . . . , bk Þ =

n X i=1

2 ð^ei Þ =

n X

ðyi − bo − b1 x1i −    − bk xki Þ2

(5:3)

i=1

The linear regression method is available within Microsoft Excel using the LINEST function. The syntax for this function is presented as follows: LINEST ðknown ys, ½known xs, ½const, ½statsÞ

(5:4)

When using this function to analyze a completed conjoint survey such as presented in Table 4.2: The known_ys are the scores in the right-hand column. The known_xs are the values in the N × m matrix that contains the levels for each of the criteria. The const term is an optional logical value that can be used to specify whether to force the regression constant b0 to zero. For our purpose of using the linear regression to develop criteria weights, this term is set to FALSE thus forcing b0 to zero. The stats term is an optional logical value for specifying whether to return additional regression statistics beyond the bk coefficients. When this term is set to TRUE the regression returns a full suite of additional regression statistics. A

5.2 Conjoint Surveys

135

discussion of these regression statistics is provided in Section 5.2.5. For our purposes of using the linear regression to develop criteria weights the stat term is set to TRUE to obtain the full suite regression statics.

5.2.5 Calculating Criteria Weights Figure 5.1 presents the criteria weights based on the decision makers/stakeholders alternative scores presented in Table 5.2 and the linear regression results. The figure indicates that the decision makers/stakeholders order of preference from the largest to the smallest are annual jobs (43.2%), local economic impact (23.9%), community perception (20.4%), and finally annual green energy to the local community (12.5%). Criteria Weights 20.4% 43.2% 23.9% 12.5%

Annual Jobs Annual Green Energy to Local Community Local Economic Impact Community Perception Figure 5.1: Criteria weights from Table 5.2 conjoint survey.

The formula for calculating each of the criteria weights is presented as equation (5.5). This equation indicates that the weight for each individual criterion is simply the value of the coefficient for the criterion divided by the sum of all the coefficients: Bi wi = Pk

i=1

Bi

(5:5)

5.2.6 Interpreting Linear Regression Results This section is provided for those interested in understanding more about linear regressions as applied to criteria weighting and evaluating the results in terms of

5 The Evaluation Process – Building the MCDM Model

136

– – –

The overall quality of the regression. Whether the observed relationship between the dependent and independent variables occurs by chance rather than representing an actual relationship Whether a linear relationship exists between the independent variables xk (i.e., the value measures) and the dependent variable yi (i.e., the alternative score).

Those not interested in such a detailed discussion may wish to proceed past this section. When the LINEST function is set to produce the full suite of regression statistics, it returns a table of values as illustrated in Table 5.3. Table 5.3: Regression statistics produced by MS Excel LINEST function. b

b

b

b

b

SE R F SSreg

SE SEy df SSresid

SE

SE

SEb

The first row provides the coefficients of the regression, b0,. . ., bk. Note that these coefficients are listed in reverse order from the way they are used in equation (5.2). The second row is the standard error (or standard deviation) associated with each regression coefficient. The regression coefficients b0 , b1 , . . . , bk are estimates of the true coefficient values typically denoted as B0 , B1 , . . . , Bk . The regression coefficients are the result of a particular experiment or sampling. Therefore, if our conjoint survey was performed many times, each time with a different set of participants, we would likely obtain different scoring of the alternatives, which would in turn result in a change in the calculated regression coefficients. One of the basic assumptions of the regression is that the regression coefficients are random variables and that they are normally distributed. Therefore, the values in the second row represent the standard errors, SE0 , . . . , SEk with means of b0 , . . . , bk . The R2 in the third row is the coefficient of determination. It is often interpreted as the percent of the variation in the outcome variable (i.e., yi ) that is explained by the regression equation. The R2 quantify varies from zero to one. A value of 0.90 or 90% would mean that regression equation explains 90% of the variability in the outcome variable yi . To keep things simple, it’s helpful to know that a higher value of R2 regression is typically “better” because it means the model explains a higher percent of the variation in the outcome. The SEy symbol in the third row represents the standard error of the y estimate. A lower standard error indicates a “better” model.

5.2 Conjoint Surveys

137

The F symbol in the fourth row represents F-statistic. Below we demonstrate the use of this statistic to determine whether the observed relationship between the dependent and independent variables occurs by chance rather than representing an actual relationship. The null hypothesis for this test is that the relationship between the dependent and independent variables occurs by chance. In general, if the F-statistic is large we can reject the null hypothesis and accept the alternative hypothesis that an actual relationship exists. The question becomes a matter of how large is large. The answer is that we can reject the null hypothesis is the F-statistic is greater than F-critical. F-critical values can be found in published F-distribution tables or by the use of MS Excel’s FDIST function. We demonstrate the use of both methods in the upcoming paragraphs in relation to the regression statistics presented in Table 5.4. The df symbol in the fourth row represents the degrees of freedom. Its use in hypothesis testing is described in the upcoming paragraphs. The SSreg symbol in the fifth row is the regression sum of the squares. We will not spend time here reviewing the use of this regression statistic. The SSrsid symbol in fifth row is the residual sum of the squares. We will not spend time here reviewing the use of this regression statistic. Table 5.4 provides presents the suite of output regression statistics associated with linear regression performed on Table 5.2. Table 5.4: Output regression statistics. .

.

.

.

.

. . . .

. . . .

. #N/A #N/A #N/A

. #N/A #N/A #N/A

#N/A #N/A #N/A #N/A

We begin our review of these results by considering the coefficient of determination result which is reported in the third row as 0.99777. This is a very high value which indicates that the regression equation explains 99.8% of the variability of the outcome variable y. More simply we can say that we have a high-quality regression. Next, we review the F-statistic which is 488.21. The value that can be used to test the null hypothesis is that the relationship between the known Ys and the known Xs occurs by chance. We can reject this null hypothesis if the F-statistic is greater than F-critical. We begin by demonstrating the hypothesis test using published F-distribution tables that can be found in most statistics textbooks. Then the use of the MS Excel FDIST function is discussed. When using the F-distribution tables we begin by selecting our desired level of significance which is commonly denoted by the symbol α (or alpha) which signifies the probability level. If the null hypothesis is true, then we should only observe a value of the F-statistic greater than F-critical α% of the time. Another way to think of alpha is

138

5 The Evaluation Process – Building the MCDM Model

that it is the probability of erroneously concluding that there is a relationship when none exits. For this example, we will assume significance level of 0.05 or 5%. The next step in the process is to calculate the F-distribution numerator degrees of freedom, v1 and the F-distribution denominator degrees of freedom, v2 . We describe how to calculate these degrees of freedom but avoid a detailed description of their meaning. Those interested in this more detailed understanding of these terms are referred to any college level textbook on statistics. The formulas for calculating v1 and v2 are presented as follows: v1 = n − df

(5:6)

v2 = df

(5:7)

Note that the n in equation (5.6) refers to the number of data points. In the case of conjoint surveys, n refers to the number of rows in the survey. It should also be noted that equation (5.6) only applies when the const term in the LINEST function is set to FALSE. Applying equations (5.6) and (5.7) we obtain v1 = 4 and v2 = 4. Using these values of v1 , v2 , and α = 0.05 and an F-distribution table we find an F-critical of 6.39. Our regression F-statistic is 488.21, which is much larger than the F-critical value of 6.39. Therefore, we can reject the null hypothesis that the relationship between the dependent and independent variables occurs by chance and accept the alternative hypothesis that an actual relationship exists. Furthermore, we can conclude there is only of 5% probability (i.e., the alpha level of significance) of erroneously concluding that there is a relationship when none exits. The syntax for the MS Excel FDIST function is FDISTðF − statistic, v1 , v2 Þ. The value obtained from this is 1.48449E-5 or 0.00148%, which is an extremely small probability. Therefore, we can reject the null hypothesis that the relationship between the dependent and independent variables occurs by chance and accept the alternative hypothesis that a relationship exists between the dependent and independent variables. Note that the FDIST result is reported in the MCDM template for each of the preestablished conjoint survey designs. The last hypothesis test we will perform is to determine whether a linear relationship each independent variable xi and the dependent value yi . In other words, does a linear relationship exist between each of our value measures and the alternative score. This test is often referred to a significance test since it is used to determine if each regression coefficient is useful in estimating the value of the independent variable. To perform the significance test recall that the regression coefficients b1 , . . . , bk are estimates of the true coefficient values B1 , . . . , Bk . To determine if a linear relationship exists, we test the null hypothesis which states that the true coefficient such as B1 associated and independent variable such as x1 is zero. In other words, the null hypothesis states that there is no linear relationship between the independent variable and the dependent variable.

5.2 Conjoint Surveys

139

The symbol H0 is often used in statistics to denote the null hypothesis and the symbol HA to denote the alternative hypothesis. Therefore, the null hypothesis and alternative hypothesis for our significance test can be stated as follows: H0 : Bi = 0

(5:8)

Alternative hypothesis HA : Bi ≠ 0

(5:9)

The t-test can be used to test this hypothesis. The formula for this statistic is presented as: t=

bi − Bi sðbi Þ

(5:10)

In this equation bi is the overserved coefficient from our linear regression independent parameter we are interested in testing and Bi represents the true coefficient value that we are not able to observe. Lastly, sðbi Þ represents the observed standard error (i.e., standard deviation) associated with our observed slope coefficient. As stated in equation (5.8), for the purpose of our hypothesis test, Bi is assumed to be zero. Therefore, equation (5.10) can be reduced to: t✶ =

bi sðbi Þ

(5:11)

Note that all the information needed to apply equation (5.11) is provided in the first two rows of the MS Excel LINEST function full suite of output statistics (see Tables 5.3 and 5.4). When using equation (5.11), we reject the null hypothesis if the t-test statistic is “big” (in absolute value). However, like using the F-statistic, the question when using the tstatistic becomes one of how big is big. A general rule of thumb is to reject the null hypothesis and conclude that a significant relationship exists if the absolute value of the tstatistic is greater than 2 [6]. To demonstrate the use of the t-statistic we will begin by demonstrating the hypothesis test using published t-distribution tables that can be found in most statistics textbooks. The use of the MS Excel T.INV.2T function will then be discussed. For our example, we will use regression coefficient and standard error associated with our Annual Jobs value measure, i.e., b1 and sðb1 Þ The values are 0.47469 and 0.03449, respectively (see Table 5.4 and recall that the LINEST function reports the coefficient and standard deviation values in reverse order). To apply equation (5.11), we simply divide b1 by sðb1 Þ to obtain a value of 13.010, which is well above the rule of thumb t-value of 2. To test our null hypothesis, we perform a two-tailed t-test at a significance level of 0.05 (α = 0.05). To perform the two-tailed test the alpha level is divided by two to obtain 0.025. Consulting a t-distribution table for an alpha of 0.025 and four degrees of freedom as seen on our regression results (Table 5.4), we obtain a t-critical value of 2.776. Since our t-statistic value of 13.010 is much greater than our t-critical value of 2.776, we can reject the null hypothesis and accept the alternative hypothesis that the number of Annual Jobs is useful in predicting alternative scores. In addition, we can say that there is

140

5 The Evaluation Process – Building the MCDM Model

a 95% probability that we are correct in accepting the alternative hypothesis or inversely we have only a 5% probability that we have erroneously rejected the null hypothesis. We now turn our discussion to the use of MS Excels T.INV.2 T function. This function returns the two-tailed inverse of the Student’s t-distribution. The syntax for this function is T.INV.2Tðα, df Þ. When using this function, it is not necessary to divide the desired significance level α by two since the function performs a two-tailed test. However, if one is interested in performing a single-tailed test, α must be multiplied by two. When using a significance level of 0.05 and four degrees of freedom T.INV.2Tð0.05, 4Þ returns a t-critical value of 2.776, the same value we obtained by consulting a t-distribution table. Table 5.5 summarizes the value measure weights, regression coefficients, t-statistic, t-critical values, and significance of each value measure’s regression coefficient. Note, “Yes,” means that the null hypothesis can be rejected and therefore a linear relationship exists between the value measure and alternative score. Table 5.5: Summary of value measure weights and tests for significance.

Annual jobs Annual green energy Local economic impact Community perception

Weights

t-Statistics

t-Critical

.% .% .% .%

. . . .

. . . .

Significant Yes Yes Yes Yes

A review of Table 5.5 indicates that all the value measures are significant. In this case we can feel confident that all our value measures are important and that we have derived objective value measure weights as a function of our decision-makers’/stakeholders’ subjective preferences. Now that we have provided a demonstration of the conjoint survey/linear regression approach where all the value measures are determined to be important as result of the significance test, the question that may occur to many is what should be done if the regression coefficient for one or more value measures is found to be insignificant. In the following example we provide a demonstration of how this can happen and provide recommendations for addressing this situation. Table 5.6 presents an example where a stakeholder group participated in the exact same conjoint survey as presented in Tables 5.1 and 5.2 but scored the alternatives very differently. In this case, the stakeholder group is concerned primarily about increasing the amount of annual green energy to the community. They are also concerned about community perception but to a much lesser degree than annual green energy to the community. Within Table 5.6 we see that the stakeholder group provides a score of 5 to every alternative that involved a high level of annual green energy to the community. Those

141

5.2 Conjoint Surveys

Table 5.6: Stakeholder conjoint survey – primary concern annual green energy to community. Number

Annual jobs

Annual green energy to local community

Local economic impact

Community perception

Score











.











.











.











.











.











.











.











.

that did not score high on annual green energy the community but still did well in terms of community perception received a score of 1.2. The remaining alternatives received a score of 1. Table 5.7 summarizes the value measure weights, regression coefficients, t-statistic, t-critical values, and significance of each value measure’s regression coefficient based on the stakeholder’s scoring of the conjoint survey. Table 5.7: Stakeholder’s weights and significance results.

Annual jobs Annual green energy Local economic impact Community perception

Weights

Coefficient

t-Statistic

t-Critical

.% .% .% .%

. . . .

. . . .

. . . .

Significance No Yes No No

Note that the results of Table 5.7 indicate that as expected the annual green energy to the community received the greatest weight at a level of 96.4%. Community perception has a weight if 2.8% and annual jobs and local economic impact both have a weight of 0.4%. However, based on the significance test, the only value measure that is significant is annual green energy to the community. When stakeholders answer the questions as a group, i.e., one set of answers, it will not be unusual to have insignificant coefficients and it should not be taken to be a critical issue. If an online survey is used, and there are multiple sets of answers in the regression, then it may be worth doing a deeper analysis regarding the causes. There are four possible approaches you can employ. Here we are assuming that the answers reflect a group response at an in-person workshop.

142

5 The Evaluation Process – Building the MCDM Model

The first is to review the scoring of alternatives to see if they are truly reflective of the stakeholder’s values and willingness to make trade-offs. After additional discussion, the stakeholders may decide that they are interested in placing higher scores on those alternatives that perform better on the other criteria. Doing so may increase the significance on the other criteria. The second approach is to collect more data from the shareholder group. This can be accomplished by polling stakeholders and have the score in the model reflected the average score of the group. Or, you can have each stakeholder respond individually, which will increase the sample size and the reliability of the model. An alternative approach is to use a survey design with less criteria. When this is done, the remaining criteria often take on more significance. If this were done to address the issues with Table 5.7, a three by two conjoint survey design could be used. However, the facilitators working along with the participants would have to decide which criterion to remove. In this case it would be either local economic impact or annual jobs. Although this approach can be taken, it is not favored by the authors. The third approach is to force the Annual Green Energy Community to 100% and change the weights to the other criteria to zero. If this is truly all the stakeholders’ value, then it would be best to reflect this fact in the weighting. Like the second approach, this approach is not favored by the authors. The reason being that the weight on such a criterion could be very high but not actually 100%. The fourth approach is to accept the weights as they are. If after additional discussion, the stakeholder group feels that the weights are accurate, then it is perfectly reasonable to simply use them as it is. Remember, the goal is not to generate statistically significant weights but is to generate weights that reflect the preferences of the group. Criteria with low weights are more likely to have low statistical significance, but that does not mean that the low weights should be ignored. Regardless of the actual weights, the decision analysts should assess the sensitivity of the ranking of alternatives to uncertainty about the weights. For example, they can run the MCDM model with each set of weights to see if they have an impact on the result. In most cases, they are likely to find that the same alternative remains dominant regardless of the weighting solution chosen, with only a slight change in the overall MCDM score. Lastly, as the number of criteria and criteria levels increases, it is more likely that some of the criteria will be insignificant. In addition, the use of more criteria will increase the cognitive burden on survey participants, which could decrease the reliability of the results. Therefore, we recommend using the three by two; four by two; and five by two survey designs. This is easier to do if the objectives hierarchy contains two or more levels such as presented in Figure 5.2. Figure 5.2 is a repeat of Figure 4.5 except that the criteria weights have now been included based on the process using the conjoint survey/linear regression described throughout this section. Note that a total of 17 criteria are included in this hierarchy. Although, there is no hard rule on the number of criteria that can be included in an

20.4%

16.7%

Solar Power Development NPV $ 10.0%

Contribution to Renewable Power Standards % 23.3%

Remediation Complete Calendar Year

Positive Financial/Regulatory Impacts

23.9%

Criteria Wt:

45.2%

Vegetative Cover Impacts Acres

23.3%

63.3%

Wt:

39.0%

Sensitive Species Affected #

9.7%

2.8%

Time until Groundwater Preservation of Returns to Baseline Greenfields Conditions Acres Years

3.3%

Lifecycle GHG Emissions Tons CO2

Power Transferable Contract Implementation Negotiation Process Leverage Scale 1 to 10 Scale 1 to 10 3.4% 10.0%

Positive Ecological/Environmental Impacts

Solar Feasibility Knowledge Gained Scale 1 to 10

Positive Site Owner Impacts Regulatory Agency Relationship Scale 1 - 10

Wt = 25%

Wt = 12.5%

Community Perception Scale 1 to 10

Positive Site Owner Impacts Wt = 25% Positive Ecological/Environmental

Clean Environment & Sustainable Redevelopment

Figure 5.2: Objective hierarchy with criteria weights included.

50.0%

12.5%

Wt:

43.2%

Wt:

Green Energy Local Economic for Impact Community $ kWh

Positive Community & Economic Impacts

Remediation Cost PV $

Jobs FTEs

Criteria

Criteria

Criteria

Positive Community & Economic Impacts Wt = 37.5% Positive Financial/Regulatory

5.2 Conjoint Surveys

143

144

5 The Evaluation Process – Building the MCDM Model

MCDM model, in general we believe this number should not exceed 20. This is because as the number of criteria goes up, there is a desire to place weight on every criterion, and in some cases, this may dilute the weight that is placed on criteria that are more important to the overall analysis.

5.3 Fitting PDFs to Actual Data Now that we have established a process for quantifying preferences, we can move onto quantifying uncertainties. Except for a few well-established facts regarding our decision problem, nearly all the input parameters for the MCDM model will involve some degree of uncertainty. This includes the nonfinancial criteria, such as those presented in Figure 5.2 as well as financial criteria related to the cost of implementing each alternative. In most cases these uncertainties will be addressed by fitting the input parameters with PDFs. The PDFs assigned to a given parameter can be based on actual data or estimated with the help of SMEs. Whenever, actual data is available, fitting PDFs to this data is preferred. However, it is often the case that such data is not available for the input parameter under consideration. It should be noted that the PDF for a given input parameter can and often does differ by alternative. That is they are conditioned based on the alternative that is being scored. For example, the Annual Amount of Green Energy for the Local Community will differ by alternative based on the size of the solar facility associated with each alternative. In this section we demonstrate the use of @Risk’s distribution fitting feature. This is a powerful and user-friendly tool that those building the decision model should consider using whenever actual data can be obtained for any of the input parameters.

5.3.1 Using @Risk’s Distribution Fitting Feature To demonstrate the use of this @Risk’s distribution fitting feature we begin with the annual cost data associated with performing Phase B Environmental Investigations at a total of 350 gasoline service station sites. Table 5.8 shows the first 20 data points contained with this data set. The process of fitting a probability distribution to this data begins with selecting the cost data for all 350 data points within the MS Excel worksheet that contains the data. The @Risk distribution fitting application is found within the define grouping of the @Risk ribbon tab and is accessed by left clicking on the red triangular-shaped icon with blue histogram bars inside as shown to the right. After selecting data range and clicking on the triangular fit icon a dropdown window opens that contains choices named fit, batch fit, or fit manager. Since we have selected our data, we can simply choose fit. This selection opens the @Risk Fit Distributions to Data Window as shown in Figure 5.3.

5.3 Fitting PDFs to Actual Data

145

Table 5.8: Phase B cost data. Site                    

Phase B investigation cost , , , , , , , , , , , , , , , , , , , ,

Notice that within the @Risk Fit Distributions to Data Window there is a choice regarding the type of data that is to be fit. In this case we can assume that we are dealing with a continuous sample data meaning that the data is from an underlying set that is continuous in nature, i.e., not from a set of discrete values. It is also clear that our data is not represent (X, Y) data points nor is it ordered in an increasing or cumulative fashion. After identifying our data type, we can now look at the distributions tab to select that type of distributions that will be considered during the fitting process. The distributions tab is presented in Figure 5.4. There of four settings on this tab to be addressed. The first is the type of fitting method. This setting can be left at its default, which is parameter estimation. The other choice associated with fitting method is predefined distributions. Since we are uncertain regarding the distribution that might best fit our data, it’s unlikely that we have predefined distributions that we are interested in investigating. There are settings for both the lower limit and upper limit associated with the distribution we are interested in fitting. In most cases it is best just to leave at their default settings of unsure. Regarding the Phase B cost data we could have selected a fixed bound lower level with a setting of zero. This would prevent the analysis of distributions such as the normal distribution which can extend to negative infinity. However, this option was left as unsure to increase the number of distributions to be analyzed. It is likely

146

5 The Evaluation Process – Building the MCDM Model

Figure 5.3: @Risk fit distributions to data window.

that those that can extend below zero will be rejected in the final analysis. However, such distributions may provide and interesting fit, and there is always the option to use the truncation setting within @Risk’s Define Distribution feature to prevent the sampling of negative values from such distributions. Last, regarding the Advanced Options within the Fit Distribution to Data window can be left at their defaults, i.e., no check mark on the fixed parameter and a check mark on suppress questionable fits. Note that on the distribution tab, the distributions selected to be analyzed as a possible fit to the data are checked based on the selections regarding the type of input data (continuous, discrete, etc.) and the choices selected on the distributions tab. Nearly, all cases, the recommended distributions, are more than sufficient. However, the user does have the option to use the “select” drop down but to choose all distributions or to clear the recommend distribution and manually select those distributions they are most interested in evaluating.

5.3 Fitting PDFs to Actual Data

147

Figure 5.4: Fit distributions to data, distributions tab.

The distribution fitting feature of @Risk can run parametric bootstrapping to fit PDFs to the input data. “Parametric bootstrapping is the process by which the distribution function and its parameters are re-sampled and refit to determine estimates for both parameter and fit statistic confidence intervals. When @RISK performs a fit with bootstrapping, the fitting process will determine the parameters for each distribution function and will then resample a set of data from that distribution a set number of times. These generated data sets are then refit and the results compared to the original fit to produce confidence measures of the fitted distribution’s estimated parameters and statistics” [7]. Palisade’s help resources notes that this can be a very time-consuming process and by default is disabled. Those interested in using this feature can learn more about it by going to Palisades Help Resources website at https:// help.palisade.com. The Chi-Square Binning Tab is used to configure the chi-squared test. Chi-squared is one type of Goodness-of-Fit tests or fit ranking methods. It is an alternative to Kolmogorov–Smirnov (K–S) and Anderson–Darling (A–D) tests. These and other fit

148

5 The Evaluation Process – Building the MCDM Model

ranking methods are discussed in the following paragraph. In general, regarding the Chi-Square Binning tab, it is recommended that user select “Auto” for the number of bins and set Bin Arrangement to “Equal probabilities.” The Results Tab provides the user with the option to choose the fit ranking method to be used for ranking the theoretical distributions analyzed during the fitting process. The choices are presented in Figure 5.5 and described in the following list which is taken from the Palisade Help Resources website [8]:

Figure 5.5: @Risk distribution fitting ranking methods.

–

–

– –

– –

Akaike information criterion (AIC) – Both the AIC and BIC methods use the loglikelihood function to estimate the relative quality of the fitted distribution; both take the number of parameters of the fitted distribution into account. Bayesian information criterion (BIC) – Both the AIC and BIC methods use the log-likelihood function to estimate the relative quality of the fitted distribution; both take the number of parameters of the fitted distribution into account. Average log likelihood – Average-log likelihood also uses the log-likelihood function but uses the average across the number of samples. Chi-squared statistic – The chi-squared statistic corresponds to the most common goodness-of-fit test for fitted distributions. This statistic requires binning of the data set. Kolmogorov–Smirnov statistic (K-S) – The Kolmogorov–Smirnov statistic is generally preferred over the chi-square statistic because it does not rely on bins. Anderson–Darling statistic (A-D) – The Anderson–Darling statistic is like the Kolmogorov–Smirnov statistic, but it places more emphasis on tail values. It does not rely on bins.

The Palisade Help Resources Website Fit Ranking Page states that the distinction between the ranking methods is very complex. It also states that the distinction between

5.3 Fitting PDFs to Actual Data

149

each of these methods is beyond the scope of their help page and recommends that in general the AIC or BIC methods be used unless the results of the other methods are well understood. A discussion of the distinction between the various fit ranking methods (i.e., goodness-of-fit statistics) is also beyond the scope of this book. However, we will make a few statements regarding the use of these statistics. The first is that, as noted by Vose [9]: Goodness-of-Fit statistics do not provide a true measure of the probability that that the data actually comes from the fitted distribution. Instead, they provide a probability that random data generated from the fitted distribution would have produced a goodness-of-fit statistic value as low as that calculated by the observed data. By far the most intuitive measure of goodness of fit is a visual comparison of probability distributions.

We agree with Vose that a visual comparison of the input distributions to fitted distributions is the most intuitive measure. In addition, we believe that such a comparison is the most useful measure of goodness-of-fit. The visual approach involves overlaying the input data with the density function of the fitted distribution. It also means comparing the input cumulative probability distribution with the fitted cumulative distribution. We discuss this approach below along with a comparison of the input data of fitted distribution descriptive statistics (e.g., mean, median, mode, and select probability percentiles). This is not to say that the goodness-of-fit statistics should be disregarded. Rather we believe that they should be considered within the context of more intuitive ways of evaluating the goodness-of-fit of a theoretical distribution to the input data. If a particular analyst has a strong reason for preferring one of the goodness-of-fit statistics over another, then the analyst should make use of that statistic. As for the selection of the best fit statistic, @Risk evaluates all those listed above. Therefore, the user can review the results of each statistic when considering which theoretical distribution best fits their data.

5.3.2 Selecting Which Fitted Distributions to Use In this section we review the reasons for choosing the Weibull distribution for representing the Phase B Environmental Investigation Cost data. Figures 5.6 and 5.7 compare the Phase B cost data with the Weibull cumulative distribution function and probability density function, respectively. The color legend on both figures is that blue represents the input data and red represents the Weibull distribution. Figure 5.6 was automatically generated by @Risk’s Distribution Fitting feature since the Weibull distribution ranked highest based on the AIC goodness-of-fit test (i.e., the one selected on the Fit Distribution to Data, Results Tab as indicated in Figure 5.5). A review of the other fit statistics indicated that it also ranked highest in terms of the BIC, Chi-squared, and K–S statistics.

150

5 The Evaluation Process – Building the MCDM Model

The Weibull distribution ranked second to the Beta General distribution in terms of the average log likelihood and A–D statistic.

Figure 5.6: Comparison of Phase B cost data with Weibull cumulative distribution function.

An inspection of Figures 5.6 and 5.7 indicates that the Weibull distribution appears to be highly representative of the input data. This is especially evident when reviewing cumulative distribution graphs as presented in Figure 5.6. The Weibull distribution fits the input data so closely that it nearly covers the input data curve. Based solely on Figures 5.6 and 5.7, our intuition is that we have a very good fit. However, to increase our overall confidence in selecting the Weibull distribution, we can perform additional analysis by comparing the input data descriptive statics (mean, median, mode, and select probability percentiles) to the fitted distribution descriptive statistics. In addition, @Risk provides to other graphs that can be used to evaluate our overall fit, i.e., the probability–probability (P–P) and the quantile–quantile (Q–Q) plots.

5.3 Fitting PDFs to Actual Data

151

Figure 5.7: Comparison of Phase B cost data with Weibull cumulative distribution function.

Table 5.9 provides a tabular comparison of the input data to the fitted Weibull distribution regarding the minimum, maximums, mean, mode, median, and a suite of percentiles. In comparing the input data to a fitted distribution, usually we begin by comparing the mean values. In the case, we see that the means associated with input data and the fitted Weibull Distribution are 144,464 and 144,519, respectively. The numerical difference between these two values is $55 and the percent difference is only 0.04%. From a practical perspective the mean values of the input data and fitted Weibull distribution are the same. After reviewing the mean, we typically like to review the standard deviation since this is an indication of the dispersion of the data about the mean. Typically, we would like to see the standard deviations of the input data and the fitted distribution as close as possible. The input data has a standard deviation of $93,289 and the fitted Weibull distribution has a standard deviation of $93,181. Therefore, the standard deviation for the Weibull distribution is $108 or minus 0.12% less that of the input data.

152

5 The Evaluation Process – Building the MCDM Model

Table 5.9: Tabular comparison of input data to Weibull distribution. Statistic

Input Data

Weibull Distribution

Numerical Difference Weibull-Input

Percent Difference

Minimum

$,

$,

−$

−.%

Maximum

$,

+Infinity

N/A

N/A

Mean

$,

$,

$

.%

Mode

≈$,

$,

≈$,

≈.%

Median

$,

$,

$,

.%

Std Dev

$,

$,

−$

−.%

%

$,

$,

$

.%

%

$,

$,

$

.%

%

$,

$,

$,

.%

%

$,

$,

$,

.%

%

$,

$,

−$,

−.%

%

$,

$,

−$,

−.%

%

$,

$,

−$,

−.%

Percentiles

Like the mean values, from a practical perspective the standard deviations of the input data and fitted distribution are the same. So far comparison of the means and standard deviations from the two distributions provides further confirmation that the fitted Weibull distribution represents the input data very well. The other measures of central tendency include the mode and the median. @Risk has provided an estimate of the mode for the input data of only $38,565 and calculated mode for the fitted Weibull distribution of $76,091. The numerical difference between these two numbers of approximately 37,526 is nearly double the estimated mode of the input data. Normally, this would be of concern. However, a review of the tallest bar for the input data on Figure 5.7 indicates that the mode of the input data is likely somewhere between 50,000 and 100,000 and there is good reason to assume that it is somewhere near 75,000. Therefore, we can disregard that @Risk approximation of the mode for the input data and based on visual inspection assume that it is closer to 75,000 and very close to the fitted Weibull distribution mode. A comparison of the medians of the input data and fitted Weibull distribution indicates that the Weibull distribution exceeds the input data median by approximately $4,000. This value represents difference of plus 3.21% which is not significantly large. In general, we would like to keep the percent differences between the various statistics

5.3 Fitting PDFs to Actual Data

153

between the input data and the chosen fitted distribution within a range of ±5%. However, this is just a good practice but not an absolute rule. Looking at the remainder of the percentiles we see that except for the 10% percentile which has a difference of plus 6.04%, all the percentiles for the fitted Weibull distribution are within ±5%. Last, we review the minimum and maximum values. In terms of the minimum, we see that the Weibull distribution’s minimum is $659, approximately 5% below that of the input data. This is reasonably close and within an acceptable range. Note that there is a definite difference between the maximum of $465,101 for the input data and plus infinity for the Weibull distribution. This is due to the fact that the Weibull distribution is unbounded in the positive direction. However, if we review the 99% percentile for the Weibull distribution, we notice that this value is approximately $433,000. Therefore, values greater than $433,000 will be sampled only one percent of the time with the probabilities getting ever smaller the greater the value is above the 99% percentile. On the one hand, in terms of being conservative and allow more risk to be included into the final output of the model, it is good to leave this distribution unbounded. However, if one is concerned that inordinately large values could be sampled from this distribution, @Risk’s truncate feature can be used to place a cap on the magnitude of the value that can be sampled. We have nearly exhausted our analysis of the goodness-of-fit of the Weibull distribution. However, the last items to review are the P–P and Q–Q plots. The P–P plot is a method of graphing the cumulative distribution of the input data set (x-axis) against the cumulative distribution for the fitted distribution. The closer the distribution fits the input data, the closer the graph will be to a straight line. Figure 5.8 presents the P–P plot of the input data versus the fitted Weibull distribution. As seen on this graph the Weibull distribution forms a near straight line indicating a very good fit. The Q–Q plot compares the input data set (x-axis) to the fitted distribution (y-axis). Like the P–P plot, the closer a Q–Q plot’s graph is to a line (where x = y), the better the fit of the fitted distribution. However, unlike the P–P plot, this comparison uses the quantiles of each distribution. Figure 5.9 presents the Q–Q plot input data versus the fitted Weibull distribution. Note that on this graph the line is straight for the most part to the point that the input distribution gets to it maximum value of approximately $465,000. Here we see that the value for the fitted quantile is about $550,000. This increase is the result of the fact that right tail of the fitted Weibull distribution stretches to infinity. Therefore, as the values of the input data get larger, the fitted distribution equivalent values begin to diverge (increase) from the input data. To address this problem or at least minimize its effect, one could consider truncating the fitted Weibull distribution at a reasonable upper limit. This is what was done when this distribution was used in the portfolio model described in Chapter 3.0 where this distribution was truncated at $1 million (see Section 3.5.3.2 and Figures 3.13 and 3.14).

154

5 The Evaluation Process – Building the MCDM Model

Figure 5.8: P–P plot of Phase B investigation cost versus fitted Weibull distribution.

Figure 5.9: Q–Q plot of Phase B investigation cost data versus fitted Weibull distribution.

5.4 Defining Input Distributions Based on Expert Judgment

155

Given the P–P and Q–Q plots and all the other information presented in this section, we can confidently assume that the Weibull distribution represents or inputs data very well. The process can seem quite time-consuming given all the information provided in this section, but it goes quite fast; perhaps 10–15 min per distribution. In addition, every graph and table discussed in the section can include any number of distributions to allow for a side-by-side comparison. This feature was not demonstrated here as it is rather straightforward. Lastly, regarding @Risk’s distribution fitting feature: once the user has decided on the fitted distribution, the Fit Results Window includes a button named “Write to Excel.” This button can be used to paste the proper syntax for the distribution directly into the Excel cell where it is desired. This syntax for our fitted Weibull distribution based on this feature is as follows: =RiskWeibull(1.4373,145320,RiskShift(12590),Risk Name(“Phase B Investigation Cost”)) The syntax for the truncated distribution is as follows: =RiskWeibull(1.4373,145320,RiskShift(12590),RiskTruncate2(1000000),RiskName(“Phase B Investigation Cost”))

5.4 Defining Input Distributions Based on Expert Judgment It is often the case that actual data simply isn’t available for use in defining input distributions. There are several reasons why this occurs. The most common reason is that at some point in the past no one had the forethought to begin collecting data regarding the parameter in question. Another reason is that available data may no longer apply to the current situation as result of technology changes or other changes such as new laws and regulations. The third reason is that the issue being modeled is a new one-off type of project. Many individuals, upon hearing that a majority of a model’s input parameters are based on the judgment of SMEs, become concerned that low quality data will be used in the modeling effort. This concern is unfounded for the reasons provided in Section 1.2.4. They especially unfounded if: – All efforts are made to ensure that true SMEs have been engaged to assist with the input parameters in question. This means SMEs who have the necessary training, degrees, licenses, years of experience, or other qualifications that would enable them to provide informed and representative estimates. Note, it is not only successful experiences that make for good SMEs. Rather an expert is someone who has a wide range of experience related to the input parameter in question. This includes not only many successful experiences but also some failures. Such individuals are more likely to provide realistic estimates that capture the risks associated with the input parameter or strategic alternative being evaluated.

156

–

5 The Evaluation Process – Building the MCDM Model

The SMEs have been “calibrated,” i.e., trained to ensure that they are providing range estimates that are wide enough to capture the uncertainties involved with the parameter that they are estimating. It also means that they are making efforts, along with the help of the decision facilitators, to remove cognitive biases that might limit them from making more representative estimates.

In this section we will review techniques for improving estimates made by SMEs and trained groups in providing data needed to fit PDFs to uncertain input parameters and to assign probabilities to the one-time chance events.

5.4.1 Class Estimating Exercise This section describes a simple but effective excise that we have found useful in helping groups and individuals improve their estimating capabilities. The exercise addresses what we believe are the two most common errors that people make when developing estimates involving uncertainty. The first error is assuming that we know more than we do which leads to optimistic estimates in which we are overly confident. The second error is assuming that we know much less than we do and thus assuming that it is impossible to even begin estimating the parameter in question. The first error is most likely the result of one or more of the cognitive basis described in Section 3.8.4. Although it’s difficult to say exactly which of the possible cognitive biases is leading to the first error, the most common ones include anchoring, availability, and the unwillingness to consider extremes. The second error, i.e., the belief that an individual or group may have that they simply do not know enough make an informed estimate, much less provide a range around such an estimate. It is often the case that experts and even nonexperts know more than they think they do and if provided with additional “conditioning” information they can begin focusing on representative estimates. Regarding the second error, there exists a curious statement that we’ve heard made more than a few times (enough that it is worth mentioning) by SMEs who are willing to provide point estimates but not range estimates. The statement is that they simply “don’t have enough information to provide a range about their point estimate.” This is the opposite of what might be expected. In cases where one has little information upon which to base their estimate, we might expect that they would prefer to provide a wide range regarding the estimate and not focus on any one number within that range. If the statement about the ability to provide a point estimate but not a range estimate was made primarily made by inexperienced individuals, it would not be of concern. However, it is often made by experienced, educated, and highly intelligent individuals. The reason for the unwillingness to provide range estimates is worthy of research. Since we have not engaged in such research, we can only speculate regarding possible reasons. One possible reason is that individuals making the statement have

5.4 Defining Input Distributions Based on Expert Judgment

157

in mind several conditioning factors leading them to their point estimate. As such, they become anchored to the point estimate and are having difficulty in envision best- and worst-case conditioning factors that would lead them to estimating minimum and maximum values. Therefore, they would like additional data upon which to base their range estimates and without it, they are unwilling to provide the estimates. In developing their point estimate, these individuals were likely thinking as Bayesians. However, to develop their range around this estimate they now prefer to think as frequentists and are unwilling to make statements about a possible range without a database to draw upon. However, if no such database exists, their best approach would be to continue thinking as Bayesians and begin seeking (or at least imagining) conditioning factors that would help them in establishing reasonable best- and worst-case estimates. We’ve been involved with the exercise described below in three different ways: – as attendees to training sessions provided by a fortune 500 oil and gas company and also a training session provided by Palisade Corporation; – as training session leaders; and – as framing meeting facilitators, leading a group of individuals having previously received training involving the use of this exercise. As result of these experiences, we can attest to the effectiveness of the exercise. There are three important points to be made regarding this exercise. The first is that the exercise is essentially a “debiasing” exercise. As such, it does not focus on identifying the type of cognitive biases involved, and it simply helps the exercise participants to avoid the two previously described errors. These errors may be the result of any number of possible cognitive biases. The second point is that research suggests that a single debiasing intervention can effectively produce immediate and persistent improvements regarding six cognitive biases as follows [9]: – Bias blind spot – perceiving oneself to be less biased by than one’s peers – Confirmation biases – gathering information and interpreting evidence in a manner confirming rather than disconfirming a hypothesis being tested – Fundamental attribution error – attributing the behavior of a person to dispositional rather than situational influence – Anchoring – overweighting the first information primed or considered in subsequent judgment – Overreliance on representativeness – using the similarity of an outcome to a prototypical outcome to judge its probability – Social projection – assuming others’ emotions, thoughts, and values are like one’s own This is not to say that all the above biases are addressed by the following exercise. However, we do believe that it can address confirmation bias, anchoring, and overreliance on representativeness as well as other biases not tested such as overoptimism,

158

5 The Evaluation Process – Building the MCDM Model

availability, and unwillingness to consider extremes. The second point, and important result of the debiasing research, is that significant effects can be realized immediately following such training and that the training persists for some time into the future. The exercise we describe below is attributed to David Vose [10]. We provide a summary overview of the exercise here and have modified the questions suggested by Vose. In addition, for reasons of brevity, we left out some of the detailed information provided by Vose regarding the exercise. However, the exercise as we describe here is essentially the same as that described by Vose. The exercise involves having each of the participants provide a practical minimum, most likely, and practical maximum estimates for a number of quantities (usually within the range of 5–8). The questions regarding the quantities are chosen such that they are obscure enough that the group would not have exact knowledge of their values, but familiar enough that they are able to formulate such estimates [12]. In addition, the participants are asked to select their minimum and maximum values such that there is about a 90% chance that the true value is between them. Note that this instruction can be interpreted to mean that the participants are being asked to estimate the 5% percentile and 95% percentile meaning that there is only a 5% chance that the actual value is below their minimum estimate and a 5% chance that the actual value is above their maximum estimate and a 90% chance that the actual value is between their minimum and maximum estimate. It should be noted that with the advent of modern smart phone technology, the participants could easily perform an internet search to answer the various questions. Therefore, they must agree not to use their phone, or other electronic devices, to answer the questions since doing so defeats the purpose of the exercise. When setting up the exercise, the decision facilitators are free to choose whatever set of questions they believe provide a reasonable balance the previously described conditions of obscurity/familiarity regarding the participants involved in the exercise. Examples of such questions include: 1. Distance between Chicago and Paris in miles 2. Number of countries in the world 3. Diameter of the moon in miles 4. Amazon’s net sales in the fourth quarter of 2021 5. Mozart’s age when he composed his first symphony 6. Height of the Empire State Building from ground level to the tip of its antenna in feet 7. Gestation period for a baby giraffe in months 8. Passenger capacity of a Class III Boeing 747-8 Airliner Although it is not necessary that the reader knows the answers to these questions for the purpose of explaining this exercise, the answers to the eight questions are provided as follows: 1. 4,130 miles 2. 195

5.4 Defining Input Distributions Based on Expert Judgment

3. 4. 5. 6. 7. 8.

159

2,159 miles $137 billion Nine 1,454 feet 15 months 467

The challenge for the participants is to provide minimum and maximum values such that the range between them (i.e., the 90% confidence interval) is neither too narrow nor too wide. Range estimates that are too narrow, such that few, if any, of the actual values fall within the estimates, are a sign of overconfidence. Ranges that are excessively large such that it would be practically impossible for the actual answer to not fall within that range (e.g., estimating that Mozart’s age at the time he composed his first symphony with a minimum of 0 and a maximum of 150) is an indication of under confidence. In general, range estimates that are too narrow are much more common than those that are too wide. There are exceptions in which certain individuals intentionally create extremely wide ranges to ensure their ranges capture the actual value. However, such individuals are not taking the exercise seriously and their results should be discarded. Vose notes that, if the participants in the exercise were perfectly calibrated, i.e., their perceptions of the precision of their knowledge were accurate, there would be a 90% chance that each true value lies within their minimum and maximum estimates [12]. Since there are eight quantities to be estimated; Vose further notes that the participant’s score, i.e., the number of actual values that will fall within the range provided by each participant, can be estimated by a binomial (8, 90%) distribution as presented in Figure 5.10. Figure 5.10 indicates that there is approximately a 43% chance that a participant will answer all eight questions correctly, i.e., providing a range that incorporates the true value. Furthermore, there is a less than a 4% chance that a perfectly calibrated estimator would achieve a score of 5 or less. Vose indicates that in over 80 classes where he has performed this exercise, he has very rarely seen a score higher than a 6 [13]. Furthermore, Vose notes that, in his experience, the average score is 3. Using the average of three correct answers and assumption that the distribution of the participants test scores is approximately binomial, Vose demonstrates that this information can be used to estimate the real probability encompassed by the participants’ minimum and maximum range. This is done by first noting that the mean of a binomial distribution is np with n representing the number of trials and p representing the probability of success. Here, the number of trials is eight (i.e., the number of questions), and the probability of success p can be calculated as p = 83 = 0.375 or 37.5%. As explained by Vose, the participants believed that they were providing minimum and maximum values where there was a 90% chance of the actual value falling between those two values; however, they were actually providing a range where there is only a

160

5 The Evaluation Process – Building the MCDM Model

Figure 5.10: Binomial (8, 90%) distribution.

37.5% chance that the actual value is between their estimated maximum and minimum [14]. In other words, their range estimate is far too narrow. Figure 5.11 shows a binomial distribution involving eight trials and 37.5% probability of success. Within this distribution we see that the number three has the highest probability. Given the binomial (8, 37.5%) distribution presented in Figure 5.11, it should not come as a surprise that a total of 3 is the most sampled value from this exercise. In addition, we can now see why a total of six correct answer is so rare given a 2.9% chance of occurring. In addition, eight correct answers would be extremely rare since a probability of 0.02% represents a 2 in 10,000 chance of occurring. We should now discuss what it means if the participants are providing minimum and maximum estimates that are far too narrow. In essence, it means that they are overconfident in their estimating ability. Initially, the participants may not interpret this result as overconfidence but rather they didn’t have enough information to estimate a wider range. We have seen individuals become anchored on a particular value and when pressed to provide a range about that value they will say something along the lines of “ok let’s go with plus and minus ten percent.” When it’s pointed out that such a statement indicates that they are very confident of their base value, they often ask what we mean. A statement that is useful at this point is, “well would you be willing to bet something like $10,000 that the actual value is within this range.” Most will say no, that to address their uncertainty about the range, and their unwillingness to lose the bet, they would significantly widen their range.

5.4 Defining Input Distributions Based on Expert Judgment

161

Figure 5.11: Binomial (8, 37.5%) distribution.

5.4.2 Documenting Expert Elicitation Results Once an SME (or group) has been introduced to the exercise presented in the previous section, they are now ready to begin developing representative range estimates. However, before doing so there is one more step in the process that needs to be discussed. That is step of recording conditioning or key factors that would force the parameter to the minimum end of its range as well as those factors that would draw them to the maximum end of their range. Table 5.10 presents an example of what we have typically used for documenting the results of an expert elicitation session for cost (or revenue) input parameter pertaining to a particular alternative. Within this table, note that there are cells for populating not only the cost of the particular input parameter but also the year that the cost will be incurred (start year), and in the case that the cost element requires more than one year to complete, the duration of the actvity. Lastly, the table includes cells for documenting the key factors that would drive the cost, start year, and durations toward their minimum, most likely, and maximum values. When using a table such as this, we recommend that the facilitators never start with the most likely value. They should begin with either the minimum or maximum value. This is to prevent the SME from becoming anchored on the most likely value. In addition, prior to recording a particular value, the facilitators should seek to draw out of the SME key factors that would drive the input parameter cost (or start/duration) toward its minimum value or maximum value. Once these factors have been recorded, the SME is then asked to provide their best estimate of the minimum or

162

5 The Evaluation Process – Building the MCDM Model

Table 5.10: Expert elicitation documentation – cost input parameter. Cost  Cost

Start Year

Duration

Notes / Comments

Minimum Most Likely Maximum Minimum Cost

Most Likely Cost

Maximum Cost

Key Factors

maximum values given these conditioning or key factors. Once the minimum and maximum values have been established, the SME should then be asked to provide a list of key factors associated with the most likely value and then estimate the most likely value. It should be noted that the MCDM template has been prestructured to include total of five alternative strategies and each alternative includes a total of 20 cost (or revenue) input parameters with associated tables such as presented in Table 5.10. In addition, each alternative includes entry tables for 10 annual operations and maintenance (O&M) cost factors. Lastly, the template has been structured that each alternative has a total of 10 risk-event tables for recording the not only minimum, most likely, and maximum costs for each risk event but also their probability of occurring. Each alternative has an associated cash flow model connected to the table used to document the input parameters. Like the financial inputs, the MCDM template includes tables for documenting nonfinancial input parameters associated with each of the five strategies. Also, like the cash flow model, each strategy also has prestructured MCDM model that draws on the nonfinancial parameters.

5.4.3 Shaping PDFs Based on Subject-Matter Expert Elicitation There are two primary PDFs used for shaping input PDFs based on SME elicitation. These include the Triangular and the PERT distribution. The shape of both distributions is described by minimum, most likely, and maximum values. In addition, both distributions were first introduced in Section 3.5.7 and summarized in Table 3.3. In this table we noted that in general we avoid use of the Triangular distribution, and that the PERT distribution is our preferred distribution for use with data based on SME elicitation.

5.5 Estimating the Probability of Discrete Events

163

The Triangular distribution as the name implies is triangular in nature with the vertices of the triangle described by the minimum, most likely, and maximum values. Our biggest objection to the use of this distribution is that it has an odd shape that is not found in data sets associated with phenomena in nature or economics. To some this may seem like a minor issue but if our goal is to make our models as representative as possible, input distributions that do not appear in nature or economics should be avoided. Another and perhaps more problematic reason cited by Vose is that the mean of the Triangular distribution is overly influenced by its minimum and maximum values [15]. We prefer the PERT distribution because depending on the minimum, most likely, and maximum values selected, this distribution can look normal, lognormal, and be skewed left or right. Such distributions are much more common in nature and economics. In addition, the mean of the PERT distribution is four times more sensitive to the most likely valuable than the minimum and maximum values [16]. For these reasons we prefer the use of the PERT distribution over the Triangular distribution. There are cases when SMEs can provide minimum and maximum values but truly struggle with identifying a most likely value. When this is the case, the uniform distribution is recommended. When the value is between the minimum and maximum values is discrete, the discrete uniform distribution is recommended. Lastly, in some cases other distributions such as the cumulative, discrete, and general distributions can be valuable to represent data from SMEs. These distributions are briefly described within Table 3.3. Information on these distributions is also available using the help feature of @Risk. Last, David Vose provides a comprehensive discussion of each of these distributions, and many others, in Risk Analysis, A Quantitative Guide in both the first and second editions.

5.5 Estimating the Probability of Discrete Events As with the fitting PDFs to data, if there exist actual data regarding the likelihood of certain events, then that data should be used to establish the discrete probabilities. However, in many instances no such data exists, and individuals must estimate such probabilities based on intuition and gut feel. In some ways estimating the probability of discrete chance events without prior data can seem easier than coming up with range estimates perhaps because only one number needs to be established. However, in other ways, it can be more difficult since it often seems more subjective than estimating ranges. However, there are several techniques that are useful in making the process less challenging. These include: – The probability wheel – Standardized probability phrases and tabular visual aids

5 The Evaluation Process – Building the MCDM Model

164

– –

Reference to processes where probabilities are well known Use of Bayes’ Formula

5.5.1 The Probability Wheel Of all the methods that can be used to help SMEs estimate discrete events, the probability wheel is the easiest to use. It is particularly effective especially with those who are more visually oriented. The probability wheel is simply a pie chart consisting of two areas: one that represents the probability of the event happening and the other of the event not happening. When using the probability wheel the SMEs are asked to imagine it as a spinner in a game of chance. The wheel is set up within an MS Excel spreadsheet and the facilitator working in conjunction with the SME, or of group participants, simply adjusts the probability of the event happening as suggested by the SME or group participants. The participants are then asked to view the wheel and decide if it reflects their intuition about the probability of the event happening or not. It is often the case that the probabilities get changed a number of times as individuals view the wheel and discuss the likelihood of the event happening. Figure 5.12 presents an example probability wheel.

30%

70%

Event Happens

Doesn't Happen

Figure 5.12: Example probability wheel.

Although the probability wheel can seem a bit simplistic, it has helped many people visualize different probabilities and arrive at one that they believe best represents the situation they are facing. When using the probability wheel, the facilitators should

5.5 Estimating the Probability of Discrete Events

165

document the various statements (i.e., conditioning factors) that the SMEs are making that regarding their assessment of the probabilities.

5.5.2 Standardized Probability Phrases and Tabular Visual Aids This second method for estimating the probability of discrete events makes use of standardized probabilities phases combined with tabular visual aids. This method provides individuals with a standardized language discussing probabilities as well as an image for visualizing probabilities. This is a method that we have adopted from David Vose as described in his book Quantitative Risk Analysis [17]. Although Vose describes its use in helping an individual with estimating probabilities, we’ve have found it useful when working with groups and helping them to use the same language and have the same image in mind while thinking about probabilities. The method begins by offering the individual, or group, a list of probability phrases. The following is a list provided by Vose: – almost certain – very likely – highly likely – reasonably likely – fairly likely – even chance – fairly unlikely – highly unlikely – very unlikely – almost impossible These phrases are ranked in order with the highest likelihood at that top. In our application of this process, we then ask the individual or group is then asked to match these phases with the tabular images or trays presented in Figure 5.13. These trays represent the probability of randomly selecting one of the blue-colored balls from the tray if blindfolded. Note this image is modified from Vose [18]. Note that there is a total of 10 phrases and 15 trays. Therefore, the individual or group performing the exercise will not make use of all trays. They will simply match the phrases to those trays they feel best represent the phrase. When this is done within a group, from that point forward, the phrases and associated trays will be used to standardize the way they speak about probabilities, or at least those having values between one and 99%.

166

5 The Evaluation Process – Building the MCDM Model

1%

30%

75%

5%

40%

80%

10%

50%

90%

20%

60%

95%

25%

70%

99%

Figure 5.13: Visual aid in estimating probabilities. (Reprinted, with modification, with permission, Vose. D., Risk Analysis, A quantitative guide, 2nd. Ed., p. 288, Copyright 2000, John Wiley & Sons, Ltd, Baffins Lane, Chichester, West Sussex PO19 IUD, England).

5.5.3 References to Processes Where Probabilities Are Well Known The probability wheel and tabular visual aids are very helpful when the probabilities being discussed are between 1 and 100, but they aren’t as helpful when estimating extremely low probabilities, i.e., extremely low probability events. In this case it is often helpful to reference processes where the probabilities are well known. Below

5.6 Structuring the MCDM Model

167

are some examples. Note that the first two examples are suggested by Parnell et al. [19] in the Handbook of Decision Analysis: – The probability of 10 heads in a row on a coin flip is roughly 1 in 1,000 – The probability of a royal flush in five-card stud poker is 1 in 65,000 – The odds of being hit by lightning is roughly 1 in 1 million (this is for average activities and not foolish activities such as playing golf during a thunderstorm where the odds would improve considerably) – The odds of winning a lottery where you pick six numbers out of a pool of 49 numbers are approximately 1 in 14 million Like referencing the trays in Figure 5.13, these reference probabilities and others like them can help individuals think about extremely low probability events.

5.6 Structuring the MCDM Model This section focuses on the equations used for evaluating and ranking the decision alternatives. Much of the information presented in this section originally appeared in an article published in the Remediation Journal in an article titled “Multi-criteria decision analysis for environmental remediation: benefits, challenges and recommended practices” [21]. It is included here with permission from the publisher with some changes/additions.

5.6.1 The Additive Value Function There are a number of different methods and objective functions that can be used for ranking alternatives. However, research by Ivy B. Haung, Jeffrey Keisler, and Igor Linkov [21] involving problems where several methods were used in parallel suggest that the recommended (highest ranking) alternative does not vary significantly with the method applied. Therefore, using the additive value function which is widely recognized and easy to program within a Microsoft Excel environment is the one that we recommend. The additive value objective function is presented as follows: TVj =

I X

  wi Vi Aij

(5:12)

i=1

where TVj is the total value of alternative j, wi is the weight of value measure i, (note P w = 1), Aij is the non-normalized score for value measure i and alternative j, and  i Vi Aij is the normalized value for value measure i for alternative j. The additive value function requires normalizing the value measures, i.e., calculating   Vi Aij . There are two types of values measures: those where smaller values are preferred (e.g., cost and greenhouse gas emissions) and those where higher values are

168

5 The Evaluation Process – Building the MCDM Model

preferred (e.g., revenue, jobs created, and total mass of contaminants removed). Therefore, two normalization equations are required, i.e., equations, (5.13) and (5.14). Both equations return values between and inclusive of 0 and 1. When lower values are preferred:   Max Ai − Aij Vi Aij = Max Ai − Min Ai

(5:13)

When higher values are preferred:   Vi Aij =

Aij − Min Ai Max Ai − Min Ai

(5:14)

5.6.2 Probabilistic Normalization Since stochastic MCDM make use of Monte Carlo simulation, probabilistic normalization of value measures is required. This is performed by first determining the maximum and minimum values possible for each performance measure across all alternatives. This can be done by inspection of the input probability distributions for each performance measure. Once the maximums and minimums for each value measure have been determined, the normalization equations can be set up in the spreadsheet model. The non-normalized scores, Aij , for performance measure i and alternative j are determined by sampling the input distributions during the simulation and normalization is performed for each iteration of the model.

5.6.3 Examples MCDM Model Structure – Conceptual and Actual Table 5.11 shows a conceptual summary of the MCDM approach as it would be programmed into a Microsoft Excel spreadsheet. Note that an actual MCDM model can include many more value measures and alternatives. In addition, the values shown as the criteria/alternatives scores (i.e., Aij) are the mean values associated with the underlying PDFs. These values will change during the simulation as the PDFs are sampled. To further demonstrate the MCDM structure and the use of equations (5.12)–(5.14) we will make use of an actual MCDM project. This project involved deciding how best to manage a hard rock open pit mine that was nearing the end of its useful life. Note that to protect client confidentiality the type of metallic ore being mined as well as the name and location of the mine has been omitted. In addition, costs and revenue values have been altered from the original case although the values are within the general order of magnitude of the original case. At the time that the analysis, the mining company, was considering closing the mine earlier than its operating lease required. The reason for considering early closure

5.6 Structuring the MCDM Model

169

Table 5.11: Conceptual summary of MCDM approach. Criteria/ Total Mass Alternative COCs Removed %

Maginitude of Residual Risk

Construction Time Years

GHG Emissions Tons CO (s)

Criterioni MCDM Score

Alt



−





Ai



Alt



−





Ai



Alt





−





Ai



Altj

Aj

Aj

Aj

Aj

Aij

Altj Score

Criteria Weights

.

.

.

.

wi

l P i=1

  = wi Vi Aij

was that the highest quality ore had been mined, operating costs were increasing, and commodity prices were down. However, the current plans, prior to the decision analysis (i.e., the momentum case), called for operating the mine for another 10 years. The company was aware that some of the higher quality ore still remained at this mine but it could only be accessed via underground mining (as opposed to open pit mining). However, underground mining was cost-prohibitive at current commodity prices. If the price of the type of metal being mined reached a sufficient level at some point in the future, then underground mining could be profitable. Therefore, MCDM model was developed to analyze these three alternatives: – Early mine closure – Momentum case – Mine expansion Note that the mine expansion alternative came about as a result of the MCDM framing meeting and the use of value-focused thinking. This alternative would require investing in the underground works in preparation for higher prices. The analysis of this alternative required probabilistic forecast of commodity prices. Only five criteria were established for this project. These criteria along with their definitions are presented in Table 5.12. The non-normalized scores for each for each criterion for each alternative (i.e., the values for Aij ) is presented in Table 5.13. Note that the scores presented in this table are the mean scores. The actual Aij values change with each iteration of the model. Note that in reviewing Table 5.13 we see that all three alternatives lose money since they all have a negative NPV. However, the momentum case has the least negative NPV at minus $38 million whereas the early mine closure has NPV of minus $72 million. Therefore, from the standpoint of mean NPV, the momentum case represents a $34 million savings over the early mine closure. The additional cost associated with

170

5 The Evaluation Process – Building the MCDM Model

Table 5.12: Example actual project criteria. Criteria

Definitions

NPV

Millions of dollars ( years, inflated and discounted)

Cash Flow

Millions of dollars ( years cumulative, inflated not discounted)

Restoration

 = Self Sustaining, residential standards of clean-up  = Industrial Standards

Stakeholder Acceptance

 = Stakeholder’s pleased with overall outcome, no litigation  = Stakeholder’s deep concerns over outcome, protracted litigation/ arbitration

Plan Resolution

Months to achieve stakeholder agreement (leaseholder, regulators and community)

Table 5.13: Example actual project non-normalized scores. Non-Normalized Scores (Mean Values) Criteria

NPV ($ Millions) Cash Flow ($ Millions) Restoration Stakeholder Acceptance Resolve Strategic Plan (Months)

Strategy Fast Close

Momentum Case

Mine Expansion

− −   

− −   

−    

early closure had to do with many factors including increased restoration, demolition, and closure costs and risks of lawsuits. It should be noted, however, that later sensitive analysis indicated that if the increased cost associated with the early mine closure could be reduced, and the chance of various risk events could be lowered, and then the early mine closure would become much more attractive. We can also see from Table 5.13 that mine expansion has a negative NPV of $93 million, even more negative than early mine closure. Note that the mine expansion as a cumulative cash flow that is positive at $21 million. This has to do with the fact that the cash flow is not discounted (i.e., nominal dollars) and represents the cumulative sum over 30 years of both positive and negative annual cash flows. The cash flow eventually becomes positive in the latter years. However, it is not positive enough, soon enough to result in a positive NPV. Table 5.14 presents the normalized scores for each criterion for each alternative   combination, i.e., Vi Aij . The values in the table are the result of applying equations (5.13) and (5.14) to the values presented in Table 5.13. Note that the values calculated using equations (5.13) and (5.14) were multiplied by 100 in order to result in a normalized score that ranges from 0 to 100 rather than 0 to 1.

5.6 Structuring the MCDM Model

171

Table 5.14: Example actual project normalized scores. Normalized Score Criteria

Strategy Early Mine Closure

Momentum Case

Mine Expansion

. . . . .

. . . . .

. . . . .

NPV ($ Millions) Cash Flow ($ Millions) Restoration Stakeholder Acceptance Plan Resolution (Months)

Table 5.15 presents the weighed criteria scores and the total MCDM score for each alternative. This table indicates that, based on the total MCDM score, the momentum case is highest ranked alternative. Reviewing the individual criterion scores, we see that momentum case scores very high in terms of the Restoration and Stakeholder acceptance criteria. This indicates that the company must have placed a great deal of value on these two criteria. Table 5.15: Example actual project MCDM score. Weighted MCDM Score (Mean Values) Criteria

NPV Cash Flow Restoration Stakeholder Acceptance Plan Resolution Total

Strategy Early Mine Closure

Momentum Case

Mine Expansion

. . . . . .

. . . . . .

. . . . . .

Figure 5.14 displays the criteria weights that resulted from the conjoint survey performed for this project. Note that the highest weights are placed on stakeholder acceptance and restoration at 35% and 26%, respectively. Together these two criteria make up 61% of the total weight. This is not to say that this company is not concerned about financial performance. We have already seen that the momentum case performs best in terms of NPV as well. Furthermore, the weights placed on the two financial performance measures together sum to 31%. This is greater than the weight placed on Restoration and just slightly less than the weight placed on stakeholder acceptance. Therefore, the company is indeed concerned about financial performance. Many might ask why the company has included two different financial criteria, and furthermore, aren’t they essentially measuring the same thing. The two financial criteria

172

5 The Evaluation Process – Building the MCDM Model

Figure 5.14: Example project criteria weights.

were included because they are preferentially independent. This company has learned that using NPV as the only financial metric when considering environmental issues can lead to strategies that delay the response to such issues and that these delays often result in much larger than anticipated costs when the time comes to address them. Therefore, the company is also interested cumulative escalated cash flow which provides an indication of how costs can grow over time. Still, the company places more weight on NPV while also maintaining a focus on total cash flow. The example provides an overview how the MCDM model can be structured using equations (5.12)–(5.14). There are additional ways to analyze the output results associated with this project including a review of output cumulative distribution functions, sensitivity tornadoes, and cash flow diagram. Such results can play a significant role in the company’s decision. However, these additional results and the company’s final decision are not included here as a matter of confidentiality.

5.7 MCDM Template The MCDM template includes predesigned tables that are similar to those shown in Tables 5.13–5.15. The tables include a total of 15 criteria and 5 alternatives. In addition, the MCDM template includes tables documenting assumptions associated with nonfinancial criteria. This includes documenting factors considered when estimating each criterion’s minimum, most likely, and maximum values (or scores) associated with each alternative. This information is used to shape the PDFs for each value measure for each alternative; in other words to shape the individual PDFs that represent the nonfinancial criteria values Aij .

5.8 Cash Flow Model Template

173

5.8 Cash Flow Model Template As previously mentioned the MCDM template has been prestructured to include a total of five alternatives and each alternative includes a total of – Twenty cost (or revenue) inputs; – Ten annual O&M costs; and – Ten risk events. The cash flow model consists of two different tables for each alternative. These tables sit side by side within the same worksheet. The left-hand side table (or input table) gathers and summarizes data from the range estimating cost sheets (see, e.g., Table 5.10) for – Annual revenue – Capital costs (one-time costs) – Annual O&M costs – Future liabilities (i.e., risk events such as lawsuits, regulatory changes, and market changes) – Future liabilities O&M The minimum, most likely, and maximum cost (or revenue) estimates for each of the above items are included in the left-hand table. The table also includes default PERT probability distribution functions that make use of the minimum, most likely, and maximum values. For any one of the line items (records), the user may change the default PERT distribution to a distribution of their liking such as uniform, triangle, normal, or lognormal. The table includes a column (field) named Special Situation that can be used to indicate whether the default distribution has been changed. Each record within the Future Liabilities category contains two PDFs. The first PDF is used to model if the event has occurred. This is done using the Bernoulli distribution which returns a value of zero or one. This distribution requires the probability of the event occurring. The second distribution is used to indicate the impact of the event and is modeled using the PERT distribution as a default. Some future liabilities (i.e., risk events) involve more than a one-time cost to address the event and may involve additional operations and maintenance costs. Therefore, the model template is structured to include such costs. Table 5.16 presents a portion of the left-hand side table. This table has been condensed (by hiding rows) to show only five cost and annual O&M elements and two future liabilities. To demonstrate the use of the PDFs, values of one capital cost (i.e., plant construction), one annual O&M cost (i.e., system operation), and one future liability (i.e., lawsuit) have been included in the table. The values within the cost distribution cells represent one sampling of the distribution. Note that for this example revenue or cash inflows were not included. Had they been included these values would have been included in the model as positive values

174

5 The Evaluation Process – Building the MCDM Model

and the costs as negative values. In the case of this example, the costs were presented as positive values for purposes of simplicity. The left-hand side table of the cash flow model also includes a summary of the timing (i.e., start year) and the duration of the various cost elements. Table 5.17 shows this portion of the left-hand side table. The right-hand side table of the cash flow model is used to distribute the various costs to the years in which they will be incurred. When the model runs, the costs and timing of each line item will depend on the values sampled during each iteration. Table 5.18 presents the right-hand side table for the first 5 years for one iteration of the model (sampling event). In Table 5.18 we see that the capital costs have been equally distributed over 2 years beginning in 2024. This is in accordance with sampled start year and duration. The default model assumes an even distribution of capital costs for elements that take more than one year to complete. Although it’s very possible that cost for some elements may be unevenly distributed, for the purposes of comparing alternatives, the assumption of even distribution of capital cost will be acceptable in most cases. However, the user may update the model to account for uneven distribution of capital costs. This would require change the formula for that particular line item. Most users familiar with MS Excel formulas should be able to make this change with little difficulty. The O&M costs within Table 5.18 are also distributed in accordance with their start year. Last, the lawsuit has occurred during this iteration of the model, and the cost is incurred in 2025 in accordance with the sampled value from its distribution. The cash flow model makes use of the distributed costs to determine the net present value for each alternative. Table 5.19 demonstrates this process for years 1–5 corresponding to the cash distribution presented in Table 5.18. The first line of Table 5.19 sums up the cost elements for each year in real dollars. Note that when the model is running, the value of each cost element as well as the year they are incurred is changing with each iteration. The second line is cumulative cash flow, also in real dollars. The information in this row is seldom needed but it has been included in case this is of interest to some users. The inflation adjusted total cash flow for each year is shown in the third line (Table 5.19) and the inflated cumulative cash flow is shown in the fourth line (an inflation factor of 2.5% was input for this example). As we have seen in the mining project example, some decision makers are interested in seeing both the inflated annual and cumulative values as they provide an indication of the costs and/or net revenue they will experience each year. In addition, both revenue and costs are involved, and the mean cumulative inflated values can be graphed to provide an indication of payback period. The fifth line of Table 5.19 presents the annual discounted cash flow (a discount rate of 6% was used for this example). This involves discounting the inflated annual values shown in the third line. Since the annual values in the fifth line are both inflated and discounted, the sum of these values represents the NPV which is shown in the sixth line (note this example which involves only costs, the proper name for the

Cost 

Cost 

Cost 

Cost 









Annual O&M Cost 

Annual O&M Cost 

Annual O&M Cost 

Annual O&M Cost 









Lawsuit

Risk Event 





Future Liabilities

System Operation



Annual O&M Costs

Plant Construction



Capital Costs

Strategy 

Strategy/ Element

LS

Sys. Opp.

Plant Const.

Input Name Base (Cost)

























Special Situation

Table 5.16: Cash flow model input table cost portion.

$

$,,

$

$

$

$

$,

$

$

$

$

$,,

Min

$

$,,

$

$

$

$

$,

$

$

$

$

$,,

Most Likely Costs

$

$,,

$

$

$

$

$,

$

$

$

$

$,,

Max or StDev

$

$,,

$

$

$

$

$,

$

$

$

$

$,,

Cost Distribution

%

%

Event Probability

(continued)





Indicationliabilities

5.8 Cash Flow Model Template

175

Lawsuit

Risk Event 





Future Liabilities - O&M

Strategy/ Element

Table 5.16 (continued)

LS O&M

Input Name Base (Cost)

Special Situation

Min

$

$ $

$

Most Likely Costs

$

$

Max or StDev

$

$

Cost Distribution

Event Probability





Indicationliabilities

176 5 The Evaluation Process – Building the MCDM Model

Cost 

Cost 

Cost 

Cost 









Annual O&M Cost 

Annual O&M Cost 

Annual O&M Cost 

Annual O&M Cost 









Lawsuit

Risk Event 





Future Liabilities

System Operation



Annual O&M Costs

Plant Construction



Capital Costs

Strategy/ Element

























Min Year

Table 5.17: Cash flow model, timing of cost elements.

























Most Likely Year

























Max Year

























Year Distribution

























Year

























MinDuration Years

























Most LikelyDuration Years

























MaxDuration Years

(continued)

























Duration

5.8 Cash Flow Model Template

177

Lawsuit

Risk Event 





Future Liabilities - O&M

Strategy/ Element

Table 5.17 (continued)





Min Year





Most Likely Year





Max Year





Year Distribution





Year





MinDuration Years





Most LikelyDuration Years





MaxDuration Years





Duration

178 5 The Evaluation Process – Building the MCDM Model

179

5.8 Cash Flow Model Template

Table 5.18: Cash flow model, cost distribution. Year Count











Year











Plant Construction

$

$,,

$,,

$

$

Cost 

$

$

$

$

$

Cost 

$

$

$

$

$

Cost 

$

$

$

$

$

Cost 

$

$

$

$

$

System Operation

$

$

$

$,

$,

Annual O&M Cost 

$

$

$

$

$

Annual O&M Cost 

$

$

$

$

$

Annual O&M Cost 

$

$

$

$

$

Annual O&M Cost 

$

$

$

$

$

Lawsuit

$

$

$,,

$

$

Risk Event 

$

$

$

$

$

Lawsuit

$

$

$

$

$

Risk Event 

$

$

$

$

$

Capital Costs

Annual O&M Costs

Future Liabilities

Future Liabilities - O&M

value shown in the sixth line is present value cost. The default name NPV, which is included for users modeling both revenue and costs, was not changed for this example). Also, it should be noted that the present value of $34,548,622 is larger than the sum of the first five year present values because of additional years included in the models but not shown in Table 5.19. Now that we have described this structuring of both the MCDM model and the cash flow we have reviewed all the steps associated with the evaluation phase. We are now ready to proceed with the agreement phase, which is the focus of Chapter 6.

5 The Evaluation Process – Building the MCDM Model

180

Table 5.19: Cash flow model, net present value determination. Year Count











Year











Total Cash Flow

$-

$,,

$,,

$,

$,

Cumulative Total Cash Flow

$-

$,,

$,,

$,,

$,,

Inflation-Adjusted Total Cash Flow

$-

$,,

$,,

$,

$,

Cumulative Inflation $Adjusted Total Cash Flow

$,,

$,,

$,,

$,,

Present Value of Total Cash Flow

$-

$,,

$,,

$,

$,

Net Present Value of Total Cash Flow

$,,

5.9 Exercise This exercise uses the preestablished conjoint survey tables within the MCDM template to develop criteria weights. This can be done based on the objectives hierarchy provided in Appendix C, or it can be done for the objectives hierarchy the reader may have developed as part of the exercise given in Section 4.6 of Chapter 4. Completed conjoint surveys developed by the authors for the Appendix C objectives hierarchy are provided in Appendix E.

References [1] [2] [3]

[4] [5] [6] [7]

Havranek, T.J., Multi-criteria decision analysis for environmental remediation: Benefits, challenges, and recommended practices, Remediation, 2019, 29, 93–108. Benjamin F. Hobbs, Meier, P., Energy decisions and the environment: A guide to the use of multicriteria methods, New York, NY, USA, Springer Science and Business Media, 2000. Kacker, R. N., Lagergren, E. S., & Fillibren, J. J., Taguchi’s orthogonal arrays are classical designs of experiments, Journal of Research of the National Institute of Standards and Technology, 1991 SepOct, 96(5),577–591. Ibid. pp. 578. Ibid. pp. 577. Newbold, P., Carlson, W. L., & Thorne, B., Statistics for business and economics, Fifth ed, USA, Upper Saddle River, New Jersey, Prentice Hall, 2003, pp. 393. Fit Results Configuration, Palisade Corporation (Accessed August 18, at https://help.palisade.com/ v8_2/en/@RISK/1-Define/3-Fit/Fit-Results-Configuration.htm?cshid=21306).

References

[8] [9]

[10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]

181

Ibid. Morewedge, C. K., Yoon, H., Scopelliti, I., Symborski, C. W., Korris, J. H., & Kassam, K. S, Debiasing Decisions: Improved decision making with a single training intervention, Behavioral and Brain Sciences, 2(I), 129–140. doi:10.1177/2372732215600886. Vose, D., Quantitative risk analysis, A guide to monte Carlo simulation, first ed., West Sussex, England, John Wiley and Sons, 1996, pp. 155–160. Ibid. pp. 155. Vose, D., Risk Analysis, A quantitative Guide, 2nd ed., West Sussex, England, John Wiley and Sons, Ltd., 2000, pp. 266. Ibid. pp. 265. Ibid. pp. 266. Ibid. pp. 276. Ibid. pp. 273–277. Ibid. pp. 287–288. Ibid. pp. 288. Parnell, G. S., Bresnick, T. A., Tani, S. N., & Johnson, E. R. (2013). Handbook of decision analysis, Hoboken, NJ, USA, John Wiley & Sons, Inc, 2013, pp. 237. Huang, I. B., Keisler, J., Linkov, I., Multi-Criteria Decision Analysis in the Environmental Sciences: Ten Years of Applications and Trends, Science of the Total Environment, 409, 3578–3594.

6 The Agreement Phase As outlined in Chapter 4, the agreement phase is made of three process steps: – Develop output results – Communicate insights – Commit to implement Chapter 4 emphasized that the commit to implement is a critical component of the MCDM process. This point was also emphasized in the overview of the decision quality chain (see Figure 3.23, Section 3.9.6). The reason for this emphasis is that without a commitment to use the results of the MCDM process; resources and stakeholder trust will have been wasted. Implementing any decision will ultimately require the support of numerous stakeholders. For example, in our case study, it is clear that the successful implementation of a Sustainable Greenville strategy would require the buy-in of (or at least acceptance by) many groups. And if the MCDM results are not the cornerstone of the final decision, then implementation will be problematic. Because of the importance of the commit to implement step, we emphasized in Chapter 4 that an MCDM should always include a decision review board (DRB) and establish a decision executive (DE) and that the DRB and DE should agree to: – Clearly articulate the role of all stakeholders in the MCDM – Define how the MCDM will be used in the final decision making – Participate in several meetings that will take place during the course of the MCDM process – Fully engage with the process by reviewing meeting preread materials and actively participate in group discussion during the meetings – Commit to implement the most compelling alternative that is consistent with the values, objectives, and preferences of the DE/DRB For those technical projects where the DE, DRB, analytical team, and implementation team are all part of the same organization, a compelling alternative is all that’s needed for the commitment to implement (i.e., the decision) to be made. This is how decisions are made within large organizations that have developed a culture of decision quality. The organization will consider the issues, and potential impacts of external stakeholders, but external stakeholders are usually not part of the MCDM process. Although all the processes, tools, methods, and best practices in this book will work very well within a single organization, they can also be used to include the values, objectives, and preferences of stakeholders outside of the organization. On the one hand, this process complicates things because there is more interaction and involvement with the various stakeholder groups. Although the MCDM process is designed to structure this interaction and make it more efficient and less time-consuming, the biggest payoff https://doi.org/10.1515/9783110765861-006

6.1 Addressing the Issue of Distributed Authority

183

of this approach is that the alternative chosen by those empowered to make the decision has a high probability of being implemented without encountering significant roadblocks put in place by various stakeholder groups such as lawsuits, permit delays, and protests designed to prevent implementation entirely or significantly alter the approach. Such roadblocks can be extremely costly and time-consuming far exceeding the money and time invested in the MCDM process with increased levels of stakeholder involvement.

6.1 Addressing the Issue of Distributed Authority Our case study is centered around the desires of the mayor and city council to create A 2030 Plan for a Sustainable Greenville with a focus on maximizing the environmental, social, and economic benefits to the citizens of Greenville. This is a lofty goal, but funding a study and putting plans in place to strive for such as goal is well within the authority of the mayor and city council. It could be said that this is the job that the citizens voted them in office to perform. Together the Mayor and the City council certainly have the power and the authority to fund a study for a sustainable Greenville including hiring any needed facilitators, analysts, and subject-matter experts needed to complete the study. They also have the power and authority to choose which alternative identified by the study that they wish to pursue. Appendix D contains a strategy table that might have resulted from the mayor and city council working in conjunction with various project stakeholders, with each group working at an appropriate level of involvement based on the stakeholder analysis (see table in Appendix A, note more on the stakeholder analysis in the following section). In order to implement any of the alternatives contained in the strategy table, the mayor and city council is going to need approval from other entities that they do not have authority over. For example, the alternative named balanced development includes the following components (i.e., strategic choices): – Riverwalk and commercial development of the Green River Shoreline Development – Residential Development of the Lakefront Area – Hotspot dredging, transportation, and offsite Disposal of Green River PCB Impacted Sediment – Solar power development including 20 inland turbines In order to implement this strategy, the mayor and city council will require: – The EPA to identify hotspot dredging, transportation, and offsite disposal of Green River PCB Impacted sediment as its preferred alternative within the record of decision – Green Lakes Wind Power, or perhaps another solar development company, to be willing to install onshore rather than offshore turbines

184

–

6 The Agreement Phase

Approval from the State Department of Environmental Quality to permit inland wind turbine development

In addition, to these approvals, the mayor and city council are going to require at least reluctant acceptance from: – Friends of the Green River regarding only partial sediment removal rather than full dredging of PCB sediments from Green River and commercial development along the shoreline. However, this group is getting a riverwalk out of this alternative and reduced disruption of Piping Plover Habitat since wind turbines will not be placed along the shore. There will still be some disruption due to residential development – Grow Greenville Advocacy who will get commercial development of the river front but no casino at the lakefront – The Native American Nation who believe total dredging of PCB sediments from Green River is necessary

6.2 Case Study of Stakeholder Involvement The best hope for the mayor and city council to achieve agreement on A 2030 Plan for a Sustainable Greenville is to involve the various stakeholders in the MCDM process at the appropriate level of involvement. Determining the appropriate level of stakeholder involvement is the purpose of stakeholder analysis exercise at the end of Chapter 4. Appendix B contains an example of the stakeholder analysis exercise as might be completed by the mayor and city council working in conjunction with the decision analysis facilitators. The level of involvement assigned to each stakeholder group and their role in the process is described in the following sections. It should be noted that no group was assigned to the lowest level involvement of Inform Only.

6.2.1 Consult Level The stakeholder holder groups assigned to the consult level include: – Citizens of Greenville – Friends of Green River – Local Native American Nation Groups assigned to the consult level will be involved in the process in the form of surveys, focus groups, and town hall meetings. They will have the opportunity to provide feedback on the criteria or value measures that will be used to evaluate alternatives. In addition, they will have the opportunity to participate in conjoint surveys to help

6.2 Case Study of Stakeholder Involvement

185

identify the weights that they would place on various criteria as a group. The promise to these stakeholders is that decision makers will listen to their concerns and provide feedback about how their input influenced our decision.

6.2.2 Involve Level – – – – –

Green River Potentially Responsible Parties (PRPs) Great Lakes Wind Power Grow Greenville Business Advocacy Group Union of Concerned Developers State Department of Environmental Quality

The stakeholders included in this group will have the opportunity to participate in collaborative workshops and deliberative forums. Like the stakeholders in the consult group, they will have the opportunity to identify and weigh the criteria. They will also have the opportunity to provide input on the strategic choices that should be included in the various alternatives. However, they will not have the opportunity to actually create the final alternatives that will be scored regarding the various criteria. The promise to this group is that their concerns are directly reflected in the development of the alternatives.

6.2.3 Collaborate Level The only stakeholder assigned to the collaborate level of stakeholder involvement is the U.S. EPA. The reason for this assignment is that the sediment remediation alternative selected by the EPA for inclusion in the record of decision will have a significant impact on the plan for a sustainable Greenville. A important question regarding this assignment is will officials from the EPA Regional Office and the EPA Remediation Project Manager be willing to collaborate with the mayor and city council on the MCDM project. It is likely that they would not want to provide input on criteria or criteria weights since their approach to evaluating alternatives contained in the PRP’s feasibility study (FS) is typically not presented in such a quantitative fashion. However, they might be willing to assist with formulating solutions, that is, providing thoughts and suggestions on the alternatives contained in the strategy table. In collaborating with the mayor and city council, the U.S. EPA could become more informed of community’s acceptance of the various sediment remediation alternatives that are part of the FS being developed by the PRP group. It should be noted that alternatives that make it through the FS screening process meet the CERCLA threshold criteria of overall protection of human health and the environment and compliance

186

6 The Agreement Phase

with all applicable relevant and appropriate standards. There are other balancing criteria by which alternatives are evaluated in the FS and considered by the U.S. EPA. The remaining criteria are known as modifying criteria that include State acceptance and community acceptance. By collaborating with the mayor and city council the EPA could have a very good understanding of community acceptance of the various sediment alternatives and consider this information when selecting the alternative to be included in the record of decision.

6.2.4 Empower Level of Stakeholder No stakeholder group other than the mayor and the city council has been included at the empower level. If a stakeholder group, other than the mayor and city council, had been assigned to this group, the mayor and city council would have had to agree in advance to implement the alternative identified by what would essentially be a decision-maker/stakeholder partnership. Ultimately, obtaining agreement on the plan and the MCDM results is going to depend on two things. The first is that the stakeholders feel like they had a voice in the process, that their concerns were heard, and were used to shape the overall plan. The second is that MCDM results are communicated in a way that is both understandable and compelling. In Section 6.4, we discuss communicating MCDM results, in particular, the type of graphs and tables that we’ve found to be particularly helpful in communicating insights. Before doing so, we discuss setting up a model within @Risk in order to run it produce the desired output results.

6.3 Developing Output Results The first step in developing output results requires indicating which calculated cells within the spreadsheet model are outputs of interest. In general, for the purposes of MCDM, there are three primary output results: – Total MCDM score for each alternative – Criteria or value measure scores for each alternative – Net present value (NPV) or present value (PV) cost for each alternative Output cells of interest within the spreadsheet model are identified within @Risk by using the Add Output icon within the Model group of the @Risk tab. Note that the @Risk tab becomes part of the MS Excel Ribbon when @Risk is installed. The screenshot of the button is shown to the right:

6.3 Developing Output Results

187

Upon clicking Add Output icon, a popup window appears that can be used to name the output. It is important to name all outputs and to use concise easily recognized names. If the user does not specify an output name @Risk defaults to a name based on the row name and column name (in that order) as they appear in a table where the cell that is being chosen is located. These names are often much less concise than we would like. It is possible to produce output graphs that account for a range of values such as a cumulative cash flow over time. These summary graphs can include the mean cash flow over time, for each alternatives, as well as probability bands around the mean such as the standard deviation and the 5th and 95th probability percentiles. This type of output range is created when a set of cells is highlighted in a worksheet and the Add Output button is clicked. Once all the output cells have been selected, the user is ready to choose the model settings. This is done by clicking on the Settings icon within the Simulation group of the @Risk tab:

Figure 6.1 shows the view of the settings window when General tab is selected and Figure 6.2 shows a view of the settings window when the Sampling tab is selected. Regarding Figure 6.1, a subject that often comes up for discussion is how many simulations to run. As a general practice we tend to use either 5,000 or 10,000 iterations. This is a general preference as the number of iterations affects the length of time it takes to run the model. However, it is important to be sure that the model has converged meaning that additional iterations will not significantly affect (i.e., within a set convergence tolerance and confidence level) output statistics such as the mean for all identified output parameters. We’ve found that for a vast majority of models convergence is achieved well before 5,000 iterations. To check how many iterations a model requires to achieve convergence the user can select Automatic for the number of iterations and enable convergence testing within the convergence tab. In addition, the user can select their convergence tolerance and confidence interval. However, the default values of 3% and 95% are quite sufficient. Once these settings have been established the user can then start the simulation and it will stop automatically once convergence is achieved and the number of iterations will be displayed. The user may then choose to rerun the simulation with a setting of 5,000 iterations for reporting results using round numbers. In such a case since convergence has been confirmed at a lower number, the user can be assured that convergence is present at the higher number of iterations as well.

188

6 The Agreement Phase

Figure 6.1: @Risk simulation settings, sampling tab.

Figure 6.2: @Risk simulation settings, general tab.

6.4 Communicating Insights

189

Figure 6.2 shows our recommended default settings. In terms of sampling type, we recommend Latin Hypercube since it is a type of stratified random sample of input distributions. It ensures that the full range of the input distributions is represented in the model outputs. If Monte Carlo is selected there is a chance that the full range will not be represented. However, this is unlikely especially in the case where convergence has been achieved in the outputs. All other settings, except for the initial seed, on this window are defaults and recommended for most applications. The user can learn more about each of these default settings by using @Risk’s help feature. The initial seed setting however is important because if the model uses a random seed instead of a fixed seed, each time the model is run, it will produce slightly different results (although all within reasonable tolerance and confidence levels). However, this is undesirable for the purpose of reporting. Therefore, we recommend use of an initial fixed seed. The number we tend to use is 1618 which is based on the golden ratio except with the decimal point between the 1 and the 6 left out.

6.4 Communicating Insights The decision makers will require compelling results to commit to implement. In addition, the stakeholders will require compelling results in order to accept the decision makers’ chosen alternative. However, to find the results compelling both the decision makers and stakeholders need to understand the results. It is the role of the decision analysts to provide output results in the form of graphs and tables that the decision makers and stakeholders can understand. This does not mean that the output graphs and tables will not require additional explanation from the decision analysts. However, we believe that it is the responsibility of the analysts to present the results as clearly as possible and to explain them in the form of in-person presentations and documented reports. In nearly all cases where someone is struggling to understand the output results, it is because the results have not been provided in a way that is aligned with the individual’s preferred way of acquiring information (visually, numerically, written word, verbally). In this section we provide example output graphs and tables that we’ve found most useful for communicating insights and results from an MCDM modeling effort. The graphs and tables provided within this section are from a variety of different projects. Most of the output graphs presented here can be produced after running the MCDM Monte Carlo simulation model, selecting the cell that contains the output of interest and then clicking on the @Risk Explore icon located in the results group. Once this is done, the user may then select the type of graph they are most interested in viewing. When comparing the results with different alternatives on the same graph, the user simply selects the Add Overlay icon in the graph window and selects the cell containing the output they wish to include in the graph. The tables we’ve included here can be obtained (after running the MCDM Monte Carlo model) by clicking on the Reports

190

6 The Agreement Phase

icon in the results group of the @Risk tab and then clicking on the Summary statistics icon. Please note that we have not included model results from the Chapter 2 case study here. Those can be found at the book website https://www.degruyter.com/docu ment/isbn/9783110765861/html. Located at this link is a written report documenting the results and the working case study model.

6.4.1 Output Cumulative Distribution Functions and Probability Distributions Figure 6.3 presents output cumulative distribution functions (CDFs or risk profiles) for MCDM score and Figure 6.4 presents output probability distribution functions, also for MCMD score. The legend for both of these graphs is that the red represents Alternative A, blue represents Alternative B, and green represents Alternative C. 100% 90%

Cumulative Probability

80% 70% 60% 50% 40% 30% 20% 10% 0% 10

20

30

40

50

60

70

MCDM Score Alt. C

Alt. B

Alt. A

Figure 6.3: MCDM score cumulative distribution functions.

When interpreting cumulative distribution functions for MCDM score, we prefer curves that a further to the right and move vertical in nature. Further to the right indicates a higher score while more vertical in nature indicates that there is less risk or uncertainty in achieving this score. Therefore, based on Figure 6.3, Alternative C is the preferred alternative. Recall that the MCDM score is a measure of how well an alternative achieves or is aligned with our values, objectives, or preferences.

6.4 Communicating Insights

191

When reading cumulative distribution functions, one starts on the vertical axis and chooses one of the cumulative probability values such as the 50% cumulative probability (which is the same as the 50% percentile which is also known as the median) and moves horizontally until one of the curves is encountered. Then the user can move downward until the horizontal axis is encountered and read the associated value. Based on this approach the 50% cumulative probability (or median) for Alternatives A, B, and C is 28, 47, and 59, respectively. This 50% cumulative probability means that there is a 50% chance that the alternative will achieve a score of less than or equal to the 50% probability value and there is a 50% chance that it will exceed the 50% cumulative probability value. In Figure 6.3 the curves never cross and the green curve is furthest to the right. Therefore, regardless of the underlying uncertainty, Alternative C is always superior to the other two alternatives at every probability percentile level. This means that this alternative is stochastically superior and the decision makers can confidently select this alternative and know that it was the best choice they could make in terms of achieving their overall preferences. In some cases, two or more curves on the CDF graph will cross each other. For example, let’s say that the curve for Alternative B crossed the curve for Alternative C at the 75th percentile. If such a result were to be found the user can use the @Risk scenarios report to identify at what level the most significant input distributions would have to sample in order to provide output results for the Alternative B that are at its 75th percentile. Figure 6.4 shows the MCDM score PDFs, which provides another view of the same information presented in Figures 6.3. When comparing the output PDFs for various alternatives in terms of MCDM score, the alternative that is scoring best is the one that is furthest to the right and thinner in nature (again less uncertainty). In this case, Alternative C is again the highest ranking alternate. Whenever dollar values are included in the model either in the form of NPV or present value cost, output CDFs and PDFs like those found in Figures 6.3 and 6.4 can be produced. However, in this case dollars will appear on the horizontal axis rather than MCDM score. The approach for interpreting these graphs is the same as those for Figures 6.3 and 6.4. Therefore, CDFs and PDFs showing NPV and PV results are not presented here. We have found that some individuals prefer the look of CDFs over PDFs while for others the reverse is true, even though they essentially present the same information. As decision professionals we feel that it is our responsibility to help communicate model results as best we can, and therefore, we don’t limit ourselves to using one type of graph.

192

6 The Agreement Phase

20% 18%

Relative Frequency

16% 14% 12% 10% 8% 6% 4% 2% 0% 10

20

30

40

50

60

70

80

MCDM Score Alternative A

Alternative B

Alternative C

Figure 6.4: MCDM score probability distributions.

6.4.2 Sensitivity Tornado Diagrams An example sensitivity tornado diagram is provided in Figure 6.5. The length of the bars indicates the impact of the value measures on the mean MCDM score. The value measures having the largest amount of uncertainty and that are driving the risk in the alternative score are located at the top of the diagram. In this case, the duration of fish consumption advisories (years), mass removal, and the number of jobs gained are the higher risk elements. For example, if the risk consumption advisories were to be removed in the shortest number of years (a case where a smaller number is better), the mean score for the alternative represented by the graph would go from 48 to 56.

6.4.3 MCDM Score Stacked Bar Graph Bar graphs showing total MCDM score and the contribution of each criterion score are useful for visualizing the value of each alternative. Figure 6.6 shows a comparison of MCDM scores for three different alternatives (this is from a different project than the one having results shown in Figures 6.3 and 6.4). From this graph we can see that the Community Acceptance and Mass Removal (i.e., the amount of contamination) are the two most important criteria driving the alternative scores. Note that the height of the bars in this graph is based on the mean score for each criterion. When the model is running the height of the bars changes with each iteration. In other words, this graph is useful for showing the contribution of various criteria but not the uncertainty regarding the criteria scores.

6.4 Communicating Insights

193

Figure 6.5: Example sensitivity tornado diagram.

The staked bar graph is not an @Risk predefined output graph. This graph is created using MS Excel native functionality and a table similar to Table 5.15 (see Chapter 5). 80 70

MCDM Score

60 50 40 30 20 10 0 A Road Traffic

B Alternatives Jobs

Figure 6.6: MCDM stacked bar graph.

Mass Removal

Community Acceptance

C Time to Complete

194

6 The Agreement Phase

6.4.4 Comparing Alternative Risks Using Box and Whisker Plots Figure 6.7 shows an example of a box and whisker graph that can be used to compare the risks of how well each alternative will perform (similar to CDFs and PDFs). The line separating the blue and green boxes represents the mean MCDM score for each alternative. The top of the green box is the 75th percent percentile and the bottom of the blue boxes represents the 25th percent percentile. The whiskers represent the minimum and maximum values. The best alternative is the one that has the highest mean score after accounting for the amount of risk. This is Alternative C. It is unfortunate when communicating the expected results of technical projects, the discussion of outcome risks is often overlooked or minimized. Later, we are often surprised to learn that a particular alternative did not perform as well as we hoped. Many will assume that it was the result of poor alternative selection or implementation, when in fact we are experiencing the effects of inherent risk. 100 90

MCDM Score

80 70

60 50 40

30 20 10

0 A

B

C

D

Alternatives Figure 6.7: Box and whisker diagram for comparing alternative risks.

The box and whisker diagram is an example of a type of output that some stakeholders find difficult to understand. In fact, some decision analysts will not use this graph for communicating insights to decision makers or stakeholders. However, we have found that there are decision makers and stakeholders who understand it perfectly well. Note that this graph was created using simulation @Risk’s summary statistics report feature to obtain model results for each alternative and Lumivero’s StatTools program (which is part of Lumivero’s Decision Tools suite) to create the graph.

6.4 Communicating Insights

195

6.4.5 Comparison of Value Measures Across Alternatives Figure 6.8 presents a comparison of the mean sediment cleanup duration for the three case study alternatives. Graphs like these can be useful in communicating the potential outcomes or consequences associated with each alternative. We’ve found these to be particularly useful when communicating with project stakeholders. Within the Monte Carlo model, the durations are represented by PDFs and the sampled duration for each alternative changes with each iteration of the model. This graph was produced using native MS Excel.

Sediment Cleanup Duration - Years

6

5

4

3

2

1

0

A

B

C

Alternative

Figure 6.8: In river cleanup duration alternative comparison.

6.4.6 Output Descriptive Statistics Output descriptive statistics are useful for reporting the results of the MCDM score, value measure scores, NPV, and present value cost for each alternative. Such tables are especially valuable for individuals that prefer numerical results to graphical results. The @Risk summary statics feature can be used to produce a wide range of output statistical results. We’ve found that it is best to keep the number of statistical results to a minimum and typically report the mean, standard deviation, and the 10th, 50th, and 90th percent percentiles. Some decision makers and stakeholders like to see other values as well such as the 1, 5, 95th, and 99th percent percentiles. These are all available in @Risk statistical output summary report. Table 6.1 presents output statistics regarding the present value costs for competing alternatives.

196

6 The Agreement Phase

Table 6.1: Alternative present value cost descriptive statistics. Statistic

Mean Standard Deviation % Percentile % Percentile % Percentile

Alternative Cost $ Millions A

B

C

    

    

    

6.5 Commit to Implement Assuming that the results are sufficiently compelling, the decision makers should now be ready to commit to implement. In addition, they should be ready to share their decision and analysis with the rest of the stakeholder groups. Lastly, they should be able to explain to the stakeholders how the input they provided helped to shape their decision. This does mean that all stakeholders will agree with the final decision but the process of involving them in the MCDM should increase their willingness to accept the decision. Regardless of the final decision that is made, we believe that the MDCM process will go a long way in creating a shared vision and increase the likelihood of successful implementation.

6.6 Summary Statement Our objective has been to provide a structured process for improving decision-making surrounding technical projects that impact a large number of stakeholders, involve many complex options, each with its own level of technical and financial risk. By using the MCDM approach we’ve sought to help identify strategic alternatives that are aligned with decision makers’ and stakeholders’ values, objectives, and preferences. We’ve sought to provide guidance regarding stakeholder involvement so that the stakeholders can have a voice in the process and to increase the likelihood of acceptance, if not agreement with the final decision. Lastly, we have sought to provide a normative process that seeks to remove cognitive biases while leveraging the value of emotions and facilitate decision making based on quantitative results. We are aware that these are lofty objectives and difficult to achieve. However, as defined in the beginning of the book, objectives, as opposed to goals, are something that we strive for in the hope of continuous improvement.

6.6 Summary Statement

197

We continue to learn about the MCDM process and refine it over time. We believe that it, along with the Capitals approach, has much to offer area sustainability and livable communities. We hope that this book will inspire others to seek ways to apply MCDM and conduct research for improving the overall processes. Perhaps that objective is more realistic, and if that is the result of our efforts, it will be more than enough.

Appendix A Example Stakeholder Survey Sustainable Greenville 2030 Stakeholder Survey The city of Greenville is committed to developing Sustainable Greenville 2030 an action that will describe our vision and goals for improving the quality of life, and the environment for all our citizens. You are invited to participate in a series of stakeholder meetings to develop that plan. This brief survey collects information about your views concerning development of Sustainable Greenville 2030. The results will be presented at our stakeholder kick-off meeting. The survey will take only 10–15 min to complete, and all individual answers will remain confidential. We look forward to hearing from you! 1. What three words or short phrases come to mind when you hear, “Sustainable Greenville 2030”?

2.

3.

Please indicate which answer best reflects how you feel about the following statement: “Greenville residents can develop a sustainability strategic plan that we can all support.” □

Strongly agree

□

Agree

□

Disagree

□

Strongly disagree

Please indicate which answer best reflects how you feel about the following statement: “Virtually all the contaminated sediments must be removed from the Greenville River in order for Greenville to have a sustainable future.” □

Strongly agree

□

Agree

https://doi.org/10.1515/9783110765861-007

200

4.

5.

6.

Appendix A Example Stakeholder Survey

□

Disagree

□

Strongly disagree

Please indicate which answer best reflects how you feel about the following statement: “Renewable wind energy for residents and industry can contribute significantly to our economic development.” □

Strongly agree

□

Agree

□

Disagree

□

Strongly disagree

Please indicate which answer best reflects how you feel about the following statement: “Developing tourism along the Greenville River could make a strong contribution to economic development.” □

Strongly agree

□

Agree

□

Disagree

□

Strongly disagree

Please indicate which answer best reflects how you feel about the following statement: “Ecological restoration projects should be a key component of our sustainability strategy.” □

Strongly agree

□

Agree

□

Disagree

□

Strongly disagree

Sustainable Greenville 2030 Stakeholder Survey

7.

201

Listed below are criteria that we could use to rank or rate alternative Sustainable Greenville 2030 strategies. The goal of this question is to assess the extent to which stakeholders agree on which criteria are most important. The results will be used to facilitate discussions about the appropriate decision criteria at the stakeholder meetings. For each criterion, please indicate whether you believe it should be included in the assessment of alternative strategies.

New jobs created Acres of restored habitat Percent of contaminated sediment removed Reduction in greenhouse gas emissions New outdoor recreational opportunities Increase in residential and commercial development Years until full benefits are realized

No need to include ○ ○ ○ ○ ○ ○ ○

Somewhat important to include ○ ○ ○ ○ ○ ○ ○

Critical to include ○ ○ ○ ○ ○ ○ ○

8.

Briefly describe what you believe will be the two most significant challenges to developing and implementing the Sustainable Greenville 2030 Plan.

9.

Briefly describe the two most important strengths that Greenville can draw upon to successfully implement the Sustainable Greenville 2030 Plan.

10. Please select the stakeholder group that you most closely identify with ○ ○ ○ ○ ○ ○ ○ ○ ○

City Council/Greenville Government Friends of the Green River Native American Nation PRP Group Renewal Energy Group Company Union of Concerned Developers State or Federal Agency Private citizen Others

Appendix B Case Study: Example Stakeholder Analysis

https://doi.org/10.1515/9783110765861-008

Seeking to install wind turbines, financial incentive

State Department of Environmental Quality

Union of Concerned Developers

Mission similar to EPA and will weigh in on selected Green River Remedy, Approval for Offshore wind

Interested in lake and riverfront development

Grow Greenville Business Advocacy Most interested in installation of wind power, most likely push for riverfront development and casino Group

Great Lakes Wind Power

Local Native American Nation

Friends of the Green River

Mission is to protect human health and the environment. Primarily selection of an appropriate sediment remediation remedy. May not be as concerned of economic impacts to Greenville Wants all impacted sediments dredged from the river. Protection of piping plover habitat. Against commercial development of waterfront Wants to see fish consumption advisories removed. May prefer to see all sediments in Green River dredged

Responsible for the removal/management of PCB impacted sediment in the Green River

Green River Cleanup Potentially Responsible Parties

U.S. EPA

Interested in jobs and a clean environment

Keep citizens of Greenville happy, Re-election in two years. Interested in casino development Keep citizens of Greenville happy. Overall economic grow the Greenville. May be less interested in casino than mayor

Issue or Stake

Citizens of Greenville

City Council

Mayor

Stakeholder

2

3

3

1

2

1

3

3

2

5

5

Level of Authority (Power)

3

5

5

5

4

5

4

5

3

5

5

Level of Concern (Interest)

4

2

3

3

2

2

5

3

3

5

5

Ability to Influence Outcomes (Influence)

3

3

3

3

2

2

4

3

2

5

5

Priority

Involve

Involve

Involve

Involve

Consult

Consult

Collaborate

Involve

Consult

Empower

Empower

Level of Stakeholder Involvement

204 Appendix B Case Study: Example Stakeholder Analysis

Appendix C Case Study Objectives Hierarchy

https://doi.org/10.1515/9783110765861-009

Value Measure Units

Value Measures

Means Objectives

Yes/No

Acres Included

Metric Tons

Restore Fish Reduce GHG Establish a Lakefront Consumption Habitat Conservation Emissions using Wind Advisories to Energy Plan for Piping Plover Baseline Levels

Increase/Improve Natural Capital

Means Objectives

Fundamental Objective

Miles

Develop Outdoor Recreation Trails

No. of Jobs

Number of New Tourism Jobs

Increase/Improve Natural Capital

Sustainable Greenville

No. of Jobs

Number of New Industrial Jobs

No. of Positve Social Media Reviews

Percent of Residents with Income above Living Wage

Increase in Community Well Asthetics of Green River Recreation Area Being within 5 Years

No. of Years

Years Until Full Benefits are Realized

Increse/Improve Produced Capital

Increase/Improve Human & Social Capital

Increase/Improve Human & Social Capital

$ Millions

$ Millions

$ Millions

Public Capital Invested Grants from Infrastructure by Private Public Sources Companies Improvements

Increse/Improve Produced Capital

206 Appendix C Case Study Objectives Hierarchy

Appendix D Case Study Strategy Table

https://doi.org/10.1515/9783110765861-010

Monitored Natual Attenuation

Combined Business and Residential Nature Preserve

Nature Trail

Riverwalk with Commercial Development Casino

Capping

Residential

Nature Preserve

Confined Disposal Facility

None

Twenty Inland Turbines

Hot Spot Dredging, Transportation and Offsite Disposal

Commercial Development

Industrial Development

Twenty Offshore Turbines

Dredging, Transportation and Offsite Disposal

None

Commercial Development

Wind Power Development

Contaminated Sediment Cleanup

Lake Front Development

Green River Shoreline Development

WorkingStrategy Table - Developed During Framing Meeting

Balanced

Nature & Recreational Friendly

Business Friendly

Working Strategy Table Legend

Example of Strategy Table Developed During Framing Session

208 Appendix D Case Study Strategy Table

Balanced Development

Nature and Recreational Friendly Residential

Riverwalk and Commercial Development

Twenty Inland Turbines

None

Dredging, Transportation and Offsite Disposal

Nature Preserve

Nature Trail Hotspot Dredging, Transporation and Offsite Disposal

Twenty Offshore Turbines

Hot Spot Dredging and Disposal in Confined Disposal Facility

Commercial Development & Casino

Commercial Development

Business Friendly

Wind Power Development

Contaminated Sediment Cleanup

Lake Front Development

Green River Shoreline Development

Alternative Theme

Finalized Strategy Table

Example of Strategy Table Formatted for Clarity

Appendix D Case Study Strategy Table

209

Appendix E Case Study: Completed Conjoint Surveys and Objectives hierarchy Appendix E1 Natural Capital Conjoint Survey Natural Capital Restore Fish Consumption Advisories to Baseline Levels

Establish a Reduce GHG Develop Outdoor Lakefront Habitat Emissions using Recreation Trails Conservation Plan Wind Energy for Piping Plover

Units

yes/no

Acres Included

Metric tons

Miles

Description

 = Yes  = No

 =   = 

 = , =

 =  =

Really Good Outcome









Not So Good









Outcome Scenarios

Scoring Definitions  = Highest Possible Score  = Lowest Possible Score

Number



Restore Fish Consumption Advisories to Baseline Levels 

Establish a Reduce GHG Develop Outdoor Score Lakefront Habitat Emissions using Recreation Trails Conservation Plan Wind Energy for Piping Plover 





.











.











.











.











.











.











.











.

https://doi.org/10.1515/9783110765861-011

212

Appendix E Case Study: Completed Conjoint Surveys and Objectives hierarchy

Criteria Weights 15%

21%

32% 32%

Restore Fish Consumption Advisories to Baseline Levels Establish a Lakefront Habitat Conservation Plan for Piping Plover Reduce GHG Emissions using Wind Energy

Develop Outdoor Recreation Trails

213

Appendix E2 Human and Social Capital Conjoint Survey

Appendix E2 Human and Social Capital Conjoint Survey Human & Social Capital Number of New Tourism Jobs Units

Number of New Industrial Jobs

Asthetics of Green River Recreation Area

Increase in Community Well Being within  Years

Number of jobs Number of Number of Jobs positive social media reviews (current = )

Description  =   = 

 =   = 

 = ,  = 

Years Until Full Benefits are Realized

Percent of Years residents with income above living wage (current = %)  = %  = %

 =  years  =  years

Really Good Outcome











Not So Good











Outcome Scenarios

Scoring Definitions  = Highest Possible Score  = Lowest Possible Score

Number

Number of New Tourism Jobs

Number of New Industrial Jobs

Asthetics of Green River Recreation Area

Increase in Community Well Being within  Years

Years Until Full Score Benefits are Realized













.













.













.













.













.













.













.













.













.













.













.













.













.













.













.













.

214

Appendix E Case Study: Completed Conjoint Surveys and Objectives hierarchy

Criteria Weights 6%

12%

26% 27%

29%

Number of New Tourism Jobs Number of New Industrial Jobs Asthetics of Green River Recreation Area Increase in Community Well Being within 5 Years Years Until Full Benefits are Realized

215

Appendix E3 Produced Capital Conjoint Survey

Appendix E3 Produced Capital Conjoint Survey Produced Capital Criteria

Capital Invested by Private Public Infrastructure Grants from Public Companies Improvements Sources

Units

$ Millions

$ Millions

$ Millions

Description

 = $ million  = $ million

 = $ million  = $ million

 = $ million  = $million

Really Good Outcome







Not So Good







Outcome Scenarios

Scoring Definitions = Highest Possible Score  = Lowest Possible Score

Number 

Capital Invested by Private Public Infrastructure Grants from Public Score Companies Improvements Sources 





.









.









.









.









.









.









.









.

216

Appendix E Case Study: Completed Conjoint Surveys and Objectives hierarchy

Criteria Weights

21%

27%

52%

Capital Invested by Private Companies Public Infrastructure Improvements Grants from Public Sources

217

Appendix E4 Integrated Capitals Conjoint Survey

Appendix E4 Integrated Capitals Conjoint Survey All Capitals Criteria

Natural Capital

Human & Social Capital

Produced Capital

Description

Overall score from natural capital tab

Overall score from human & social capital tab

Overall score from produced capital tab

Really Good Outcome







Not So Good







Qualititative Yes = 







Outcome Scenarios

Scoring Definitions  = Highest Possible Score  = Lowest Possible Score

Number

Natural Capital

Human & Social Capital

Produced Capital

Score









.









.









.









.









.









.









.









.

218

Appendix E Case Study: Completed Conjoint Surveys and Objectives hierarchy

Criteria Weights

20% 47%

33%

Natural Capital

Human & Social Capital

Produced Capital

Value Measure Weights

Value Measure Units

Value Measures

Means Objectives

Improve/Increase Natural Capital

Means Objectives Weights

Means Objectives

Fundamental Objective

33%

47%

20%

Increase Improve Produced Capital

Improve/Increase Human & Social Capital

Improve/Increase Human & Social Capital

Improve/Increase Natural Capital

Sustainable Greenville

Appendix E5 Case Study Completed Objectives Hierarchy with Weights

Increase Improve Produced Capital

Appendix E5 Case Study Completed Objectives Hierarchy with Weights

219

Index additive value function 167 advocacy-based approach 4 Alternative Theme 124 alternatives 15 attributes 14 Bayesians 64 Bayes’ Formula 61 behavior economics 41 benefit–risk assessment 9 blended objectives hierarchy 119 bounded distribution 74 chance events 45 coefficient of determination 136 cognitive bias 5 collaborative journey of inquiry 98 collectively exhaustive 56 complete factorial design 132 complicating factors 1 complimentary probabilities 57 concept of co-creation 116 conceptual independence 121 conditioning events 59 constrained optimization 48 continuous distribution 67 continuous numerical event 55 cost–benefit analysis 16 criteria 14 cumulative distribution function 69

fault trees 57 FDIST function 138 finite sampling space 55 framing meeting 106 framing the problem 106 free-form questions 111 frequency histogram 65 Frequentists 64 F-statistic 137 fundamental objectives 14 goals 14 good decision 12 goodness-of-fit tests 147 health technology assessment 9 high stakes 98 high-quality set of criteria 121 identifying value measures 121 independence in probability theory 59 independent events 57 inquiry-based approach 5 inquiry-based decision making 4 joint probability 56–57 linear regression 134 LINEST function 134

decision 12 decision analysis 12 decision analysis facilitators 108 decision executive 107 decision hierarchy 106 decision makers 13 decision review board (DRB) 102 definition of a system 52 design of experiments 132–133 directionality 119 discrete numerical event 55 discrete probability distribution 67 DRB 109

MCDM process 101 MCDM template 112 mean 70 means objectives 14 means objectives 119 measures of dispersion 73 median 70 metrics 14 mode 70 multicriteria decision analysis 9, 11 multicriteria decision making 1 multiobjective decision making 12 multivariate linear regression model 134 mutually exclusive 56

evaluation measures 14 expected value of a random variable 47

natural units of measure 122 nonnumerical event 55

https://doi.org/10.1515/9783110765861-012

222

Index

objective 14 objectives hierarchy 14, 117, 127, 130, 133, 142 one percent rule 101 optimization 16 orthogonal array 133 orthogonal arrays 132 parameters 14 parametric bootstrapping 147 partial derivative 134 preference independence 121 preframing meeting 110 preframing meeting online survey 110 probability distribution function 48 probability models 46 probability theory 55 probability trees 57 probability-weighted average 47 project team members 109 P–P plot 153

stakeholder management 115 stakeholders 13, 109 standard deviation 73 stochastic MCDM 12 strategic choices 43 strategic decision 43 strategy table 123 strong law of large numbers 65 structure phase 105 structured questions 111 subject-matter experts 110 Synergetics 52 system engineering perspective 51 systems modeling 51 systems thinking 51 Taguchi orthogonal arrays 132–133 Thomas Bayes 61 trade-offs 15, 127 traditional decision-making process 4 types of constraints 48

Q–Q plot 153 R. Buckminster Fuller 52 random variable 46 range 73 regression statistics 136 risk 13 sampling 55 stakeholder engagement 116 stakeholder group 5

uncertain quantity 46 uncertainties 13 value measures 14, 50, 118 value-focused thinking 14 Values 13 variance 73 Venn–Euler diagrams 44 Weibull distribution 68