Building Better Health Care Leadership for Canada: Implementing Evidence 9780773586505

Citation preview

Building Better Health care Leadership for Canada

This page intentionally left blank

Building Better Health Care Leadership for Canada Implementing Evidence Edited by terrence sullivan and jean - louis denis

Published for The Canadian Health Services Research Foundation by McGill-Queen’s University Press Montreal & Kingston • London • Ithaca

©  McGill-Queen’s University Press 2011 isbn 978-0-7735-3860-3 (cloth) isbn 978-0-7735-3875-7 (paper) Legal deposit second quarter 2011 Bibliothèque nationale du Québec  rinted in Canada on acid-free paper that is 100% ancient forest P free (100% post-consumer recycled), processed chlorine free McGill-Queen’s University Press acknowledges the support of the Canada Council for the Arts for our publishing program. We also acknowledge the financial support of the Government of Canada through the Canada Book Fund for our publishing activities.

Library and Archives Canada Cataloguing in Publication Building better health care leadership for Canada: implementing evidence / edited by Terrence Sullivan and Jean-Louis Denis. Includes bibliographical references and index. ISBN 978-0-7735-3860-3 (bound). – ISBN 978-0-7735-3875-7 (pbk.) 1. Health services administration – Canada.  2. Medical care – Canada – Decision making.  3. Medical care – Canada.  I. Sullivan, Terrence, 1951– II. Denis, Jean-Louis  III. Canadian Health Services Research Foundation RA184.B85 2011

362.10971

C2011-901611-7

This book was typeset by Interscript in 10.5/13 Sabon.

Contents

Foreword Owen Adams and Jean Rochon

vii

Acknowledgements ix Introduction: The Evolution of the EXTRA Program Jean-Louis Denis, Terrence Sullivan, Samuel B. Sheps, Nina Stipich, and Jonathan Lomas xi Section One

Why EXTRA?

3

1

The Policy Imperative: Why Better Evidence in Decision-Making? Paul A. Lamarche, Raynald Pineault, Jean Rochon, and Terrence Sullivan 5

2

The Management Imperatives for Leading Clinical Change Robert S. Bell 23 Section Two

the Curriculum Elements

37

3

extra Program Basics Terrence Sullivan, Jean-Louis Denis, and Samuel B. Sheps 41

4

Supporting Research Use in Policy and Management John N. Lavis 47

5

Building Research Literacy for Management David L. Streiner, Paula N. Goering, and Jeffrey S. Hoch 56

6

Leadership Skills for Evidence-Informed Decisions Terrence Sullivan, Margo Orchard, and Muriah Umoquit 70

vi

Contents

7 Managing and Sustaining Change Ann Langley, Karen Golden-Biddle, Jean-Louis Denis, and Trish Reay 84 8 Mentoring Adults for Research Application Samuel B. Sheps 105 9 Evaluating the First Years of the EXTRA/FORCES Program Malcolm Anderson for the EXTRA Evaluation Team 122 Section Three What Regional Leaders and Fellows Say about the Experience 151 10 Building Capacity for Change: Examples from the Winnipeg Regional Health Authority Brian Postl 153 11 The EXTRA Experience in Montérégie Denis A. Roy, Sylvie Cantin, and Stéphane Rivard 167 12 The Whole Elephant: Many Perspectives, One Transformation at a Nova Scotia Health District Chris Power 178 Conclusion: Supporting the Individual as Change Agent and the Organization as Responsive to Change Ward Flemons, Andrea Seymour, and Carl Taillon 189 Contributors

201

Foreword On Behalf of the Executive Training for Research Application (extra ) Program Partners: Canadian College of Health Leaders, Canadian Medical Association, Canadian Nurses Association, Quebec Consortium owen adams and jean rochon

Since its establishment in 1997, the Canadian Health Services Research Foundation (CHSRF) has pioneered evidence-informed decision making through collaboration between the “doers” and the “users” of health services research, in part by mandating partnerships between researchers and decision makers in the design and execution of its research grants, thereby increasing both the relevance of the research and the likelihood of its uptake. The Executive Training in Research Application (EXTRA) program is further solidifying this approach by bringing together senior decision makers from the health services delivery field with a faculty of distinguished health services researchers to work through an intensive curriculum of the application of research evidence in decision making. We foresee several enduring benefits from the EXTRA program. First, we think that the EXTRA projects conducted by the fellows will help increase the uptake of research in the health care delivery system. Second, by mandating the participation of nurses, physicians, and other health executives, EXTRA has fostered interdisciplinary teamwork; the recent solicitation of team projects from within institutions will enhance this collaboration even further. Third, we expect

viii

Foreword

that EXTRA will spawn a national community of practice that will continue to foster the exchange of knowledge and experience and that in due course may mitigate the Canadian tendency to “reinvent the wheel” across jurisdictions. Fourth, drawing the EXTRA fellows from different sectors of the health care delivery system (hospitals, community care, public health, health ministries) may help to break down the system’s long-lamented “silos.” We commend the EXTRA fellows and faculty for their tremendous effort and commitment. Because there are competing demands for people’s time and a growing array of choices for leadership training in the health sector, it has not been easy to recruit applicants to the program. However, those who have come forward are superb. Recruitment has been a constant effort for all parties, but by all accounts it has been worth it. We hope that this volume captures the essence of the richness of the exchange between the research and the decision-making communities and that it will serve as an inspiration to others to adopt this approach more widely within Canada and internationally.

Acknowledgments

The editors would like to acknowledge the contributions made by seven cohorts of fellows – and those to be made by future cohorts – to the ongoing evolution of an important training experience. The fellows provide renewed leadership in the healthcare delivery system. Thanks are also due to the lead and guest faculty who have been so actively involved and who have shaped and refined the original curriculum and methods of teaching. The leadership and staff of the Canadian Health Services Research Foundation have been the organizational linchpins of this important program from its inception to the present – in particular Nina Stipich and Jessie Checkley, who provide program leadership, coordination and execution. In addition, Jennifer Verma, Jasmine Neeson, Kerrie Whitehurst and Beth Everson have provided superb support in the coordination and production of this volume. Thanks also to the 2003 EXTRA Program Design and Development Team: Steven Lewis, Lillian Bayne, Armand Boudreau, Jack Altman, David Rochefort, Jean-Louis Denis, Nina Stipich, Pierre Sauvé and Jonathan Lomas, for their knowledge and expertise in the design phase; to the members of the EXTRA Advisory Council, chaired by Dr. Jean Rochon, for their strategic advice; and to our key partners at the Canadian Medical Association, the Canadian Nurses Association, the Canadian College of Health Leaders and the Quebec Consortium, for their active support of the program. The McGill Queen’s University Press is gratefully acknowledged for its active role in getting the story to print. Terrence Sullivan Jean-Louis Denis

This page intentionally left blank

Introduction The Evolution of the extra Program jean - louis denis , terrence sullivan , samuel b . sheps , nina stipich , and jonathan lomas

This volume is about the evolution, the content, and the early consequences of a unique fellowship program designed to improve decision making in health care organizations. The effort to strengthen research inputs in the management of health systems is not new: it has been part of the utopian dream to which enlightened states have aspired. Francis Bacon wrote New Atlantis in the seventeenth century, preaching an enlightened Utopia governed by knowledge, science, and medicine. More recently, following the significant attention given to evidence-based clinical medicine originating from McMaster University in the 1980s and early 1990s, health care management and indeed the emerging area of health services performance measurement and management is being subjected to the same scrutiny (Gray 1997). As suggested by Pfeffer and Sutton (2006) and Rousseau and McCarthy (2007), management education based on evidence promises improved managerial decision making and better organizational outcomes. Evidence in management reduces the use of ineffective management practices while making effective approaches more widespread. While the professional and multidisciplinary nature of management is less well specified than that of medicine, it is just as necessary (Walshe and Rundall 2001). Given the emerging call for health system reforms based on evidence of comparative best performance, it would seem to be indispensible (Smith et al. 2009).

xii

Introduction

What has become apparent in the last few years, however, is that using research to inform management decision making involves more than just summarizing results in more understandable aggregate forms and finding persuasive ways to communicate them. At the receiving end of these persuasive messages we find individuals, professional cultures, and organizations with their own unique context – a context that has not historically been actively receptive to research results (Lomas 2007). Moreover, the average health system manager is not well versed in where to find, assess, and employ research or comparative organizational data as part of his or her day-to-day work. The average organization is neither designed nor incentivized to routinely incorporate research into decision making. We have only recently recognized this “receptor capacity” role for management organizations as a requirement for the effective, ongoing use of research results by managers and their organizations (Zahra and George 2002; Lemieux and Champagne 2004; Denis 2005; Lomas and Brown 2009). In 2003 when the Canadian Health Services Research Foundation (CHSRF) reviewed the curricula of the health service executive-training programs available in Canada, it found that none focused on the skills needed to acquire, appraise, adapt, and apply research (Romilly 2003). This was clearly a serious, if not fatal, missing link in the chain of requirements for evidenced-informed management and performance improvement in health services. The CHSRF’s response to this gap was the Executive Training for Research Application (EXTRA) program, a fellowship program designed to improve the receptor capacity for research evidence. Indeed the program has been intentionally designed to increase the capacity within Canada’s health care organizations to make better decisions and advance performance based on evidence. Research extraction, digestion, and application in executive decision making are core competencies for the modern health service executive and the professional health bureaucrat alike. For instance, the United Kingdom’s civil service now identifies “analysis and use of evidence” as one of four core public service competencies, alongside financial management, people management, and program management (Campbell et al. 2007).

p u r p o s e a n d ov e rv i e w This volume tells the story of the EXTRA program and its components from three perspectives organized into three sections. Section I

Introduction

xiii

highilights the policy and management imperatives for such a program (chapters 1, and 2 respectively)? Section II describes some of the unique and evolving curriculum elements (chapters 3 to 8). Section III provides commentaries on how the program has influenced the fellows and the host organizations that have sponsored multiple fellows (chapters 9 to 11). Each section begins with a brief editorial comment to introduce the chapters. In this fashion, the book covers the unique curriculum and support elements of the EXTRA program from the multiple perspectives of faculty, fellows, management, and policy-makers. We encouraged the lead author of each chapter and the fellows writing commentaries to reflect critically on the EXTRA experience. We hope the volume will be read by those interested in the improvement of performance in health care organisations through training, and those interested in the curriculum elements necessary to train health executives for this purpose. At the time of writing, the EXTRA program is evolving as a learning curriculum. The first seven years suggest that there is a strong appetite for knowledge-rich, health service executive education focused on improving the application of various forms of evidence. The contributors to this book have made numerous suggestions to refine and strengthen the curriculum content and the program design, and a significant field evaluation currently under way will certainly add further to the program’s evolution. We believe EXTRA has benefited the individuals who have taken the program (as illustrated in chapter 9) by enabling them to use evidence in their own working settings. However, a more recent evaluation of the evidence of large shifts in the culture and organizational performance of their institutions by trainees of the early cohorts, while promising in certain respects, seems less compelling (Champagne et al. 2010). Early evidence presented by Scott et al. (2003) suggested that the relationship between organizational culture and performance has been hard to characterize despite some evidence of a relationship. They called for more careful specification of common operational definitions of culture and performance. However, it does seem clear that cooperative working relationships between operational management and clinicians are an important factor in improving hospital performance (Klopper-Kess et al. 2010). The EXTRA program is evolving to admit a greater number of mixed teams from a single institution seeking practical skills for

xiv

Introduction

specific organizational projects for performance improvement. Based in part on this initial organizational evaluation, we believe the teambased approach holds more promise for organizational impact on culture and ultimately performance. As we consider the directions for the future evolution of the program, we are struck by two compelling imperatives. The first is to invest in the applied science of performance and quality improvement to help health care systems cope with increasing cost pressures with a quality lens, i.e., to determine what works and how it works. Here, despite emerging interest, the evidence is actually fairly thin. We need to reinvest in our knowledge base, something the traditional granting agencies have not been strong on. Moreover the investments in this country to date on direct quality initiatives are quite meagre (Gagliardi et al. 2010) and largely purchased externally; (Sullivan et al., forthcoming). The second imperative is to aggressively expand competency-based training activity to build practical, context-sensitive, evidence-based leadership at multiple levels of management in order to move fast enough in the process of health system performance improvement. The EXTRA program has begun the process of responding to the second demand and will further refine the curriculum elements to maximize impact. The program has enjoyed the benefit of significant support from the Canadian Health Services Research Foundation, its Canadian partners, and the fellows’ home organizations. We hope that some of the initial elements of this curriculum may be useful in the context of other delivery systems. We look forward to refining and expanding the curriculum application domestically and to the benefit of continued international learning collaboration. Portions of this introduction appeared in a special edition of the Journal of Health Services and Policy Research: JeanLouis Denis, Jonathan Lomas, and Nina Stipich, Creating receptor capacity for research in the health system: The Executive Training for Research Application (EXTRA) program in Canada. Journal of Health Services Research Policy 13, S1 (2008):1–7.

references Campbell, S., S. Benita, E. Coates, P. Davies, and G. Penn. 2007. Analysis for policy: Evidence-based policy in practice. London: UK Government Social Research Unit.

Introduction

xv

Champagne, F., L. Lemieux-Charles, G. MacKean, T. Reay, J.C. Suàrez Harrera, M. Anderson, N. Dubois, and M.F. Duranceau, 2010. Knowledge Creation in Healthcare Organizations as a Result of Individuals’ Participation in the EXTRA and SEARCH Programs. Preliminary Report (unpublished). Denis, J.-L. 2005. Linking knowledge and health care organizations more tightly. Presented at the 6th International Conference on the Scientific Basis of Health Services. Montreal, Canada, September 18–20. Gagliardi, A., C. Majewski, J. Victor, and R. Baker (2010). Quality improvement capacity: A survey of hospital quality managers. Quality Safety in Health Care 19:27–30. Gray, J.M. 1997. Evidence-based healthcare: How to make health policy and management decisions. New York: Churchill Livingston. Klopper-Kes, A., N. Meerdink, C. Wilderom, and W. van Harten. 2010. Effective cooperation influencing performance: A study in Dutch hospitals. International Journal for Quality in Health Care 10.1093/intqhc/ mzq070. Lemieux Charles, L., and F. Champagne. 2004. Using knowledge and evidence in healthcare: Multidisciplinary perspectives. Toronto: University of Toronto Press. Lomas, J. 2007. The in-between work of knowledge brokering. British Medical Journal 334(129):129–32. Lomas, J., and A. Brown. 2009. Research and advice-giving: A functional view of evidence informed policy advice in a Canadian ministry of health. Milbank Quarterly 87(4):903–26. Pfeffer, J., and R.I. Sutton. 2006. Evidence-based management. Harvard Business Review 84(January):1–13. Romilly, L. 2003. Environmental scan of management training programs for health systems managers. Unpublished Report prepared for the EXTRA planning process. Rousseau, D.M., and S. McCarthy. 2007. Evidence-based management: Educating managers from an evidence-based perspective. Academy of Management Learning and Education 6:94–101. Scott, T., R. Mannion, M. Marshall, and H. Davies. 2003. Does 0rganisational culture influence health care performance? A review of the evidence. Journal of Health Services Research & Policy 8: 105–17. Smith, P., E. Mossialos, I. Papanicolas, and S. Leatherman. 2009. Performance measurement for health system improvement: Experiences, challenges and prospects. Cambridge: Cambridge University Press.

xvi

Introduction

Sullivan T., F. Ashbury, J. Pun, B. Pitts, N. Stipich, and J. Neeson. 2011. Responsibility for Canada’s healthcare quality agenda: Interviews with Canadian health leaders. HealthCare Papers, forthcoming. Walshe, K., and T. Rundall. 2001. Evidence-based management: From theory to practice in health care. Milbank Quarterly 79(3):429–57. Zahra, A.S., and G. George. 2002. Absorptive capacity: A review, reconceptualization and extension. Academy of Management Review 27(2):185–203.

Building Better Health care Leadership for Canada

This page intentionally left blank

section one

Why extra ? In chapter 1, Paul Lamarche and colleagues take on and try to set the policy context for the use of evidence in health care allocation and management decision making. They make it clear that current views of evidence, particularly of its efficacy and effectiveness, constitute only one of several competing and valued forms of knowledge that managers may consider in making health care management decisions. This first chapter lays out how different influences, such as feasibility, acceptability, and efficacy, play key roles in evaluating decisions, along with the method of engagement and dissemination and the value of different forms of knowledge. In the view of the authors, better use of evidence should implicate both researchers and decision makers, which will have the effect of democratizing the use of such knowledge. They argue that the very definition and organization of systems of health services should evolve in new and desirable ways as a consequence of better use of evidence, a commendable democratic wish. In chapter 2, Robert Bell presents the management imperative for training from the perspective of a physician CEO working inside a large academic hospital. He brings a clear recognition of the need for integration between management science and clinical/ medical science in the operation of large health care organizations. His chapter emphasizes the complementary nature of these two bodies of knowledge and the need to use both to facilitate improvement in health care delivery and clinical outcomes. He illustrates the issue elegantly, using the story of a staged implementation of Computerized Physician Order Entry (CPOE) systems in his hospital.

This page intentionally left blank

1 The Policy Imperative Why Better Evidence in Decision Making? paul a . lamarche , raynald pineault , jean rochon , and terrence sullivan

Introduction This chapter seeks to succinctly describe the policy imperative by identifying the factors that prompt the increased use of evidence in decision making. It also identifies a number of limitations to expanding the use of evidence in the organization and delivery of health services. It examines five elements: the increased need for evidence, the nature of the evidence, the relativity of the evidence, the users of evidence, and suggestions to increase its use.

The Increased Need for Evidence Several factors argue in favour of the increased use of evidence in decision making with respect to the organization and delivery of health services: the nature of the decisions, the availability and accessibility of evidence, and observers’ perception of the correctness of the decision and the credibility of the decision makers. A significant proportion of decisions about the organization and delivery of health services consists of choosing actions to solve a problem, improving a situation, or achieving an objective (Pressman and Wildavsky 1973; Heclo 1974; Majone, 1980). Measures to improve the accessibility, continuity, and quality of services, as well as the intention to control health system costs, are concrete examples of this. Each of these decisions rests, either implicitly or explicitly, on an intervention approach relating the problem, that is, the situation

6

Why extra?

and the goal sought, on the one hand, to the proposed actions and to the anticipated effects, on the other hand. It is in the best interest of decision makers and their organizations that this problem-solving approach should rest on the strongest foundation possible. Scientific evidence constitutes such a foundation. There is a growing trend towards improving the performance of entire health systems to better meet the service needs of the population (OECD 2004). Efforts are being made to measure and compare the performance of systems and their components (Health Council of Canada 2007; Commonwealth Fund 2008). One of the challenges in the coming years will be to identify the factors that influence this performance and to act on them. Scientific evidence provides an important way to address this challenge, and initiatives are already being taken in this regard (Conner-Spady et al. 2007; Sullivan et al. 2008; Commissaire à la santé et au bien-être 2009). Moreover, the availability and accessibility of scientific evidence has increased significantly in the past few years. Health services organization and policy research have long been seen as the poorer cousins of health research, and perhaps they still are. It is clear, however, that there is more research exploring various aspects of the policy, the organization, and the delivery of health services being conducted now than ever before. Many scientific journals specialize in publishing the results of research in this field, including Health Affairs, the Journal of Health Politics, Policy and Law, and Evaluation in the United States; the Healthcare Quarterly, Healthcare Papers, and Healthcare Policy in Canada; and Pratiques et Organisation des services de santé in France. The internet ensures the globalization of knowledge in this area, and e-libraries allow for a comprehensive review of the literature on a given topic – all without anyone ever leaving the office. Some groups have given themselves the mandate to synthesize research evidence (e.g., CHSRF’s Mythbusters (http://www.chsrf.ca/mythbusters/index_e.php) and Policy Syntheses (http://www.chsrf.ca/dss/ index_e.php)), and they make abstracts of their meta-analyses accessible on the web (e.g., Cochrane Collaboration (http://www.cochrane. org/reviews/) and Campbell Collaboration of Systematic Research Reviews (http://www.campbellcollaboration.org/)). Not so long ago, this scientific evidence came primarily from anglophone countries, including the United States and England and to a lesser degree Scandinavia. Now it comes from various environments, including from emerging countries such as China, Taiwan, and Singapore.

The Policy Imperative

7

Expectations regarding the use of scientific evidence are increasing significantly, which is confirmed by the growing number of publications on evidence-based decision making in health care. The use of evidence now influences observers’ trust in the intervention approach and the actions proposed by decision makers, as well as the credibility of these same decision makers. For example, talks are currently under way in Quebec regarding the public-private partnership formula advocated by the current administration for the construction of university medical centres. Opponents cite the results of foreign experience as a reason to abandon this formula (Vadeboncoeur and Goulet 2009). As for the government agency charged with its promotion, the latter provides no data in support of the anticipated benefits of this formula. Because of this, a well-known editorialist has declared this agency’s credibility to be null and void (Sansfaçon 2009). The epithets “arbitrary” and “political” are often associated with decisions perceived as not evidence-based (Peterson 1995).

The Nature of the Evidence The decisions we want to see based on evidence concern the choice of interventions in the organization and delivery of health services. It is hoped that they will be based on the proven ability of the interventions to solve a problem, improve a situation or a system’s performance, and achieve an objective; in other words, that they will be based on their effectiveness (Klein 2003), which must, preferably, be revealed through scientific research. The effectiveness that is shown constitutes what is generally referred to as “evidence” and “scientific evidence.” As noted in the introduction, the intention to base decisions on evidence originated in clinical medicine, which aims to base the choice of an intervention and, in particular, of a drug or other medical intervention, on its proven effectiveness in treating a health problem. While we recognize the desirability of using this approach in making decisions related to the organization of health services, its generalization is limited by, among other things, the very nature of the decisions to be made, the multiplicity of anticipated effects, and the influence of the context and of stakeholders on the effectiveness of interventions. Furthermore, criteria other than effectiveness are taken into account in decision making in this field. Although these criteria in no way lessen the importance of using evidence in decision

8

Why extra?

making, it is nevertheless necessary to adapt their definition, as well as the ways in which they are obtained and used, to the particular decision type and process concerned. These criteria also require a broader knowledge-source base. Interventions in the organization and delivery of health services are increasingly complex (Pineault et al. 2010). They are generally composed of several interdependent elements involving many stakeholders, and they are established within various contexts, such as care programs for the elderly with decreasing independence, privatepublic partnerships, and the decentralization of powers to regional and local authorities. Each of these interventions includes several elements that simultaneously affect governance, the allocation of resources, and the roles and powers of the various stakeholders. As well, these interventions are established within non-experimental frameworks. The people responsible for the frameworks’ implementation and the people taking part in them learn from their experience and modify and adjust the intervention with each lesson learned. Consequently, most of these interventions are difficult to define with accuracy, which makes them quite different from controlled interventions and observations in clinical studies. It is especially difficult to associate the effects observed with their various elements: are they associated with particular elements, all of the elements, or the consistency connecting one intervention to the next (Meyer et al. 1993)? The same intervention can lead to various outcomes, depending on the context within which it is implemented or on the stakeholders who implement it. The effects are not solely or even mainly dependent on the components of the interventions but more on their interaction with the context and the stakeholders involved (Meyer et al. 1993). Some of our own research on primary care also suggests that organization models produce different results depending on the context (Lamarche et al. 2009; Pineault et al. 2008). Some models seem better suited to certain contexts than others. Research syntheses rarely focus on the interactions between the interventions and the contexts. The lack of consideration regarding these interactions increases the ecological risk, which consists of attributing to each stakeholder what is observed within the group as a whole. This risk of error is often criticized in clinical practice guidelines. Health service interventions often aim for several objectives simultaneously, and none of the options achieves all the objectives at the same time. Accordingly, the choice of objectives cannot rest solely on

The Policy Imperative

9

their relative effectiveness. Because each objective may be as desirable as the others, a priority ranking must be established. This prioritization often takes place outside the area of influence of scientific evidence. Debates on environmental protection, economic development, and energy preparedness (Sabatier 1993), primary care organization (Lamarche et al. 2003), and the measures to be fostered to increase service accessibility and continuity (Pineault et al. 2005) are all routine examples of this reality. The measurement of the effectiveness of interventions cannot be based solely on the overall effects: it must also be based on the gaps between the effects, gaps that make it possible to assess the extreme, or threshold, effects associated with the interventions. Socially and politically unacceptable situations can hide behind an overall positive assessment of an intervention. These often-isolated cases, which some would qualify as anecdotal, have the capacity to bring into question an intervention that on the whole has generated positive effects. For example, the case of a patient who died on a stretcher in an emergency room after waiting for forty-eight hours without being examined by a doctor can bring into question an entire generally effective plan to free up emergency rooms. For the proven effectiveness to become the criterion of choice for health services policy and organization, equal weight must be given to the effects observed in large-scale implementation, in what is now embraced as comparative-effectiveness research. The effectiveness of interventions is rarely the only criterion influencing decisions; their feasibility and acceptability must also be considered (Klein 2003). “Feasibility” refers to the ease of implementation within a given context. It means, among other things, the availability of the financial, human, and technological resources needed or the managerial expertise required for their implementation.“Acceptability” refers to the responses that the interventions are likely to prompt within the population, in the media, and among interest groups. While the effectiveness of interventions aims to address what Peterson (1995) called programmatic uncertainty (uncertainty regarding the effects associated with the interventions), feasibility and acceptability aim to address the political uncertainty regarding the choice of intervention. Assessing interventions based on these three criteria requires the diversification of knowledge sources. Peterson (1995) identified three. First, knowledge that relies on scientific methods for its development is generally thought of as systematic, unbiased, and less

10

Why extra?

fallible than other knowledge and is generally designated as “evidence.” The CHSRF intends to increase the use of this knowledge in decision making. Second, popular or experiential knowledge, which is obtained from day-to-day experience, observation, and interaction, comes from personal experience, daily contacts, and media publication scans. It provides some insight into how health systems work, the reasons for their disruption, and the potential solutions. This knowledge, however, is seen as non-systematic, biased, and highly fallible. Finally, distributional knowledge takes into consideration the interests of the stakeholders likely to be affected by a decision, as well as the type and intensity of the potential response. This knowledge is particularly important when the implementation of measures depends on the support of stakeholder or interest groups, among which there are often winners and losers associated with the various options. While scientific evidence makes it possible to measure, in particular, the effectiveness of the interventions, experiential and distributional knowledge makes it possible to measure feasibility and acceptability.

The Synergy of Evidence Scientific evidence is not the only recognized foundation for decision making, nor is it the only one with value. Other foundations include intuition, the prevailing school of thought or ideology, and the requests of stakeholder groups. These foundations are often perceived as rivals in decision making, effectively cancelling out one another’s influence, with the result that if the influence of one is great, that of the others will necessarily be less. To us, this does not always seem true. On the contrary, these foundations can be used synergistically with scientific evidence to increase influence in decision making. In fact, one school of popular thought promotes the use of intuition in decision making (Cyr 2008; Gladwell 2005). Intuition is defined as a sixth sense or the voice inside our heads that almost inevitably points decision makers to the right decision. It is, in effect, the decision maker’s GPS. To increase its effectiveness, decision makers must believe in it and trust it. The limitations of scientific evidence provide the main argument in support of using it. Indeed, scientific evidence is not available on all decision topics: when it does exist, it is often contradictory. Furthermore, the evidence necessarily involves

The Policy Imperative

11

interventions that have been in existence long enough to have allowed for the collection of data. It is accordingly oriented towards the past rather than the future. New interventions, often perceived as original and promising, cannot be founded on evidence. For this school of thought, creativity and innovation accordingly cannot rest on scientific evidence and must rely more on intuition. A bestseller from the 1980s (Peters and Waterman 1984) provided arguments in support of this school of thought, at least implicitly. The authors identified the main characteristics of the most profitable businesses of their era. One was a focus on action founded in large measure on intuition. Scharmer’s Theory U (2007), whose popularity is still growing, sees the knowledge of evidence as one of the first steps in decision making. The appropriation of evidence by decision makers and its projection in time and space based on what this evidence inspires in them constitute vital, complementary steps in making creative and innovative decisions. Another foundation of decision making is the “prevailing school of thought,” which is conveyed by people recognized as experts in a given field and is considered true as a result of the credibility of those who convey it, not necessarily because it is founded on evidence. The more a prevailing school of thought is accepted and conveyed by serious people, the more it is considered to be true, regardless of its scientific validity. The importance of this factor is illustrated in the CHSRF’s Mythbusters series, which has attempted to sway the prevailing school of thought using scientific data on several topics associated with health services organization and policy. This series addresses, among other things, the influence of growing expenditures, user fees, institutional mergers, and non-profit institutions on the performance of the health care system. Its importance was also brought to light in the 1990s by Andersen Consulting, which interviewed close to one hundred managers of European health systems to identify the strategies most likely to mark their evolution. The introduction of user fees is one of the strategies most often cited, because it is consistent with the prevailing school of thought, even though the managers were convinced of its negative effects on users and health systems. The importance of this foundation is recognized especially in institutional theory, which sees organizations in a given field as all following a homogenous pattern under the influence of organizations considered exemplary in the field (mimicry), in spite of the scientific evidence (DiMaggio and Powell 1991). Its importance

12

Why extra?

is also recognized by Paul Krugman (1998), Nobel Prize winner in economics (2008), in his discussions of globalization and the political decisions that result from it. Ideology, which also plays a key role in decision making related to public powers, can be defined as a more or less systematic set of beliefs that define the aims of a given field, offer an understanding of how it works and the reasons for its disruption, and provide suggestions for solutions. Ideology relies on foundations other than scientific evidence: beliefs and evidence may coincide, but they may also conflict and even be opposed. If they coincide, the proponents of these beliefs can also become proponents of the evidence. If they conflict, a debate on their respective primacy is unavoidable. Tensions between beliefs and evidence have long existed in areas that go well beyond health systems. Copernicus and Galileo suffered the wrath of the Roman Catholic Church because of the heliocentric vision of the world they proposed as an alternative to the geocentric vision defended by the Church. The debate in the United States between creationism (intelligent design) and Darwin’s theory of evolution are the same. The statement recently made by the head of the Roman Catholic Church on the relation between condoms and the spread of AIDS is another. US president Barack Obama’s (2009) desire to give science back its rightful place in public policy formulation is a sure sign of the significant presence and influence of ideology in the formulation of such policies. Finally, the interests of the stakeholders involved play a key role in decision making related to health services organization and policy. Consideration of these interests is intrinsically linked to decisions on public powers (Moran 1995), which are often the result of compromises between the conflicting interests of these groups (Sabatier 1993). Some authors even feel that the true test of a good decision in areas as complex as health is not the link shown between the end and the means, which is sometimes difficult to establish, but rather the consensus that arises with respect to the necessity of the means (Lindblom 1959). The Commission d’enquête sur les services de santé et les services sociaux (Rochon 1988) described the health system as a prisoner of stakeholder groups, which give precedence to the pursuit of their own interests over the common good. Their role and influence have also been demonstrated in the evolution of health systems in a number of countries (Peterson 1995; Tuohy 1999; Moran 1999; Hacker and Marmor 2004).

The Policy Imperative

13

It must be said, however, that scientific evidence and interest groups are not always in opposition. These groups also use scientific evidence to assess the contribution of various measures to the achievement of a number of objectives. What characterizes them, however, is that they give more weight to evidence that agrees with the interests they are defending, and they may deny and challenge even the strongest scientific evidence if it is does not agree with their interests. This leads some observers to say that the value of evidence is primarily associated not with its scientific strength but, rather, with the number and influence of the stakeholders whose support it succeeds in rallying (Jordan and Maloney 1997).

The Users of Evidence Our notion of decision making with respect to health services determines who must be made aware of the scientific evidence to increase its use. The most common notion is that of the “iron triangle” (Jordan and Maloney 1997), according to which most decisions result from the interaction of a limited number of stakeholders primarily composed of the lawmaker, his or her key aides and administrators, and representatives of a few interest groups. Participation by these groups is deemed necessary because of their expertise and also to legitimize the decision process in the eyes of those who are most likely to be affected, which will thus to limit criticisms and potential conflicts. This approach suggests a definition of a target audience for scientific evidence that goes beyond health system administrators to include those who support them, as well as representatives of influential stakeholder groups in sectors such as doctors’ and nurses’ associations. This vision of decision making about the organization of health services no longer corresponds to today’s reality. In many cases, this “iron triangle” calls on outside help, resulting in a community of influence (Sabatier 1993). In the past few decades, the federal and provincial governments have called on people outside the iron triangle who have extensive expertise in this field specifically or in the formulation of public policy in general. Concrete examples of this are, at the federal level, the National Forum on Health (1997) and the Romanov Commission (2002) and, at the provincial level, the working groups and task forces chaired by Duncan Sinclair (2000) in Ontario, Jean Rochon (1988), Michel Clair (2000), and Claude Castonguay (2008) in Quebec, Don Mazankowsky (2001) in Alberta, and Ken  Fyke

14

Why extra?

(2001) in Saskatchewan. The members of these groups or commissions, as well as their aides, form part of the community of influence. Almost every day newspapers contain articles dealing with one aspect or another of the organization of health services. Many of the articles simply transmit news, but others include analyses of how the system or one of its components works, the problems it faces and their causes, and potential solutions. These analyses have a definite influence not only on popular knowledge but also on the lawmakers and decision makers in the field. The media’s influence is now so strong that it is considered a new source of power in society. The journalists and analysts who specialize in and devote themselves to health also form an integral part of this community of influence on health services organization and policy. The target audience may be further expanded to include potential stakeholders in the organization of health services – those with an interest in the field and who, with the appropriate information, are likely to become active. Activating these stakeholders is a recognized strategy in the creation of coalitions, and it is also an important vehicle for the transmission and use of scientific evidence to influence decisions. These coalitions can include patients’ associations, community organizations, health co-operatives, or even municipalities; all are increasingly concerned with the organization of health services within their jurisdictions.

Options for the Future Figure 1.1 summarizes the various components of the evidence utilization process. Consideration of these components, a better understanding of their influence, and a recognition of their importance are all likely to influence the use of evidence in the coming years. Five different possible approaches in particular come out of this analysis. They constitute a basis on which the CHSRF and EXTRA/FORCES might guide their actions over the next few years. The first approach – the expansion of the assessment criteria for interventions – is illustrated in the centre of figure 1.1. Effectiveness is a fundamental criterion. Its assessment must go beyond the average effects associated with the interventions to include the observed gaps related to these averages and to take into account the potentially mediating effect of the context and the stakeholders involved (Sheldon 2005). The effectiveness of the interventions must be complemented

The Policy Imperative

15

Production

Interest Groups Use

Effectiveness

Scientific knowledge

Evidence

Acceptability

Outreach

Feasibility

Experiential knowledge Deliberation

Figure 1.1

Evidence Use Process

by feasibility and acceptability criteria, since their influence on decision making is often interdependent. The fact that an intervention is highly effective could increase its acceptability, and greater acceptability in turn influences the feasibility of an intervention, at least in terms of policy. One can easily imagine the strength of the influence that evidence that coincides could have on the effectiveness, feasibility, and acceptability of an intervention (Lavis et al. 2004; Pineault et al. 2005). The second approach is illustrated in figure 1.1 by the elements surrounding the centre. It comprises the recognition and use of various types of knowledge as sources of evidence: scientific knowledge, experiential knowledge, and knowledge of the interests of stakeholder groups. In previous work on primary care, we experimented with attempting to combine scientific knowledge with knowledge obtained through experience. The intention was to assess the feasibility and acceptability of interventions or to determine the degree to which effects observed elsewhere could also be observed in particular contexts. The opinions of researchers and practitioners across Canada were collected using the Delphi method to find out the degree to which they trusted that the effects associated with various primary care organizational models – revealed through a systematic review of the results experienced in fourteen industrialized countries – could also be observed within the Canadian context. Highly interesting

16

Why extra?

nuances were discovered (Lamarche et al. 2003, appendix 2; Pineault et al. 2010). Another experiment, titled “The Research Collective on the Organization of Primary Care Services in Québec” (Pineault et al. 2005, 2007), was intended to synthesize the results of thirty ongoing or recently completed research projects on the organization of primary care and to identify options for their improvement. This experiment was marked by the participation of decision makers, which made it necessary to clarify some of the research results and the context within which they were obtained. It also confirmed and reinforced other results. The third approach is related to the fact that meta-analyses and systematic reviews constitute the preferred methods of producing evidence and the methods that are most likely to influence decision making (Sabatier 1993; Lavis et al. 2004). This approach is not new, but its recognition and use are growing. One indication is the financial support granted by funding agencies recognized for the production of such syntheses. Another indication can be seen in the efforts deployed by the CHSRF (Lomas et al. 2005; CHSRF 2005), other research funding agencies (Birdsell et al. 2005), and the CIHR (Canadian Institues for Health Research) to identify the best ways of producing them. The fourth approach aims to extend the temporality of evidence and consists of evaluating new interventions. However, innovations in health services organization and policy that have no connection to known interventions are few and far between. Current knowledge may make explicit the intervention approach for new interventions and allow the results to be measured. For these experiments to become evidence, they must be combined with an evaluation process. It is the experimentation-evaluation combination that is most likely to mitigate the limitations of the evidence’s temporality. The fifth approach concerns the dissemination of evidence. It consists of expanding the circle of recipients of evidence in health services organization and policy. This circle must be as large and as diversified as that of the stakeholders who want to influence decisions in this field. One concrete example is the role played by researchers in the debate on the issue of how to follow up on the Supreme Court of Canada ruling in Chaouli, a decision that opens the door to duplicate health insurance and the increased role of the private sector in

The Policy Imperative

17

the delivery of services to the population (Paradis and Robert 2009). On the Quebec Global Health Research Network’s initiative, health services organizations and policy researchers published a series of articles on this ruling, its causes, and the potential consequences for the population and the health system. They provided evidence that served as the basis for three regional debates and one Quebec-wide debate bringing together citizens interested in the reasons for and potential consequences of the private sector’s increased role in the funding and delivery of health services. According to participants, the information provided by researchers influenced and counterbalanced the prevailing school of thought on this matter. The researchers’ contribution culminated in the publication of a book on the private sector in health care (Béland et al. 2009) that was recommended to the general public by a well-known literary critic as one of four required summer reads (Cornellier 2009). These examples suggest various uses for scientific evidence to guide decisions on health services organization and policy (Pelz 1978), although the work of Lavis and his colleagues in creating a corpus of reviews in this area is truly ground-breaking (Lavis et al. 2004). This instrumental use is the one that is generally advocated. However, it rests on a notion of the nature and process of decision making in this field that, as we have tried to illustrate, does not quite seem to match reality (Black 2001; Brouselle et al. 2009). Evidence can also be used to enlighten and even define discussions on health services organization and policy. This is the so-called “conceptual” use of evidence. The two examples mentioned above fall within this view, which continues to be used to influence decisions, but in a less specific and more indirect way. Rather, it is used to influence the way members of the political community think about a particular topic and structure the debate on it. To this end, the conceptual view may be used to define the problem to be solved, determine the potential solutions most likely to provide the anticipated results, enlighten the validity of the intervention on which these solutions are based, specify their feasibility and acceptability, define the conditions to be respected to achieve the effects, and so on. It may also be used to assess the theoretical and empirical plausibility of the other solutions proposed. Expanding the circle of the recipients of the evidence, that is, the political community in the field, is even more appropriate when the evidence is to be used conceptually.

18

Why extra?

The sixth and last approach relates to the deliberations that are often required to give meaning to evidence and the resulting actions. More often than not, the evidence reveals that interventions foster winners and losers and that not all aspirations are achieved in decision making. The intervention then orients actions in different directions depending on what is considered desirable, preferable, or a priority. The sixth approach consists of creating spaces for deliberations intended to give meaning to the evidence in relation to one or more clearly defined desirable elements. The definition of these desirable elements is likely to end up at the centre of evidence utilization in the coming years.

Conclusion It is now considered normal and even desirable to democratize the production of knowledge by engaging researchers and users. It is also considered normal and desirable to democratize access to the evidence by broadening its dissemination to all those interested in the field of health services organization and delivery. To us, it also seems normal and desirable to democratize the definition of the health systems of the future. It is with regard to this contribution that the usefulness of evidence may ultimately be assessed, a contribution for which the need for evidence currently seems to us the greatest.

references Birdsell, J.M., R. Thornley, R. Landry, C. Estabrooks, and M. Mayan. 2005. The utilization of health research results in Alberta. Edmonton, AB: Alberta Heritage Foundation for Medical Research. Black, N. 2001. Evidence based policy making: Proceed with care. British Medical Journal 323:275–9. Brouselle A., D. Contandriopoulos, and M. Lemire. 2009. Using logic analysis to evaluate knowledge transfer initiatives. Evaluation 15(2):167–85. Canadian Health Services Research Foundation. 2005. Journal of Health Services Research. Special Issue 10:S1. Castonguay, C. 2008. Report of the task force on the funding of the health system: Getting our money’s worth, final report. Québec: Government of Quebec.

The Policy Imperative

19

Clair, M. 2000. Les solutions émergentes. Final report of the Commission d’étude sur la santé et les services sociaux. Québec: Government of Quebec. Commissaire à la santé et au bien-être. 2009. Rapport d’appréciation de la performance du système de santé et des services sociaux: État de situation portant sur le système de santé et de services sociaux et sur sa première ligne de soins. Québec: Government of Quebec. Connor-Spady, B., G. Johnston, and C. Sanmartin. 2007. Patient and surgeon views on maximum acceptable waiting times for joint replacement. Healthcare Policy 3(2):102–16. Cornellier, L. 2009. Pas de vacances pour les idées, Lectures d’été, Le Devoir, 6 and 7 June, F9. Cyr, M. 2008. La petite voix. Éditions Transcontinental. DiMaggio, P.J., and W.W. Powell. 1991. “The iron cage revisited: Institutional isomorphism and collective rationality in the organizational field.” In The new institutionalism in organizational analysis. Chicago: The University of Chicago Press. Fyke, K. 2001. Caring for medicare: Sustaining a quality system, Saskatchewan commission on medicare report. Government of Saskatchewan. Gladwell, M. 2005. La force de l’intuition (Blink). Éditions Transcontinental. Hacker, J.S., and T.R. Marmor. 2004. Medicare reform: Fact, fiction and foolishness. Public Policy and Aging Report 13(4). Health Council of Canada. 2007. Health care renewal in Canada: 2007 annual report. Ottawa. Heclos, H. 1974. Modern social politics in Britain and Sweden: From relief to income maintenance. New Haven, CT: Yale University Press. Jordan G., and W.A. Maloney. 1997. Accounting for subgovernments: Explaining the persistence of policy communities. Administration & Society 29(5): 557–83. Klein, R. 2003. Evidence and policy: Interpreting the Delphic oracle. Journal of the Royal Society of Medicine 96 (September): 429–31. Krugman, P.R. 1998. La mondialisation n’est pas coupable: Vertus et limites du libre-échange. Paris: La Découverte/Poche. Lamarche, Paul A., et al. 2009. L’expérience de soin : Une résultante de l’interaction des modes d’organisation et des contextes, gris. Université de Montréal. Lamarche, P.A., M.D. Beaulieu, R. Pineault, A. Contandriopoulos, J.-L. Denis, and J. Haggerty. 2003. Choices for change: The path for

20

Why extra?

restructuring primary healthcare services in Canada. Ottawa, Canada: Canadian Health Services Research Foundation. Lavis, J., F.B. Posada, A. Haines, and E. Osei. 2004. Use of research to inform policy-making. The Lancet 364:1615–21. Lavis, J., A.D. Oxman, S. Lewin, and A. Fretheim. 2009. Support tools for evidence-informed health policy-making. Health Research and Policy Systems 16(7S): I1. Lindblom, C.E. 1959. The science of muddling through. Public Administration Review 19(2): 79–88. Lomas, J., et al. 2005. Conceptualizing and combining evidence for health system guidance: Final report. Ottawa: Canadian Health Service Research Foundation. Majone, G., and E. Quade. 1980. Pitfalls of analysis. New York: John Wiley and Sons. Mazankowski, D. 2001. A Framework for reform: Report of the premier’s advisory council on health. Edmonton, ab: Government of ab. Meyer, A., S. Tsui, and C.R. Hinings. 1993. Configurational approaches to organizational analysis. Academy of Management Journal 36(6): 1175–95. Moran, M. 1995. Three faces of the health care state. Journal of Health Politics, Policy and Law 20(3): 767–81. – 1999. Governing the health care state: A comparative study of the United Kingdom, the United States and Germany. New York, NY: Manchester University Press. National Forum on Health. 1997. Canada health action: Building of the legacy, final report. Ottawa. Obama, B. 2009. This represents the largest commitment to scientific research and innovation in American history. President Obama addressing members of the National Academy of Sciences, 27 April 2009. OECD. 2004. Increasing value for money in health systems: The quest for efficiency. In Towards high-performing health systems. Paradis, G., and L. Robert. 2009. Research illuminating public policy debates: Private sector roles in Quebec healthcare. Healthcare Policy 4(3): 72–5. Pelz, D.C. 1978. Some expanded perspectives on use of social science in public policy. In M. Yinger and S.J. Cutler, eds., Major social issues: A multidisciplinary view. New York: Free Press. Peters, T., and R.H. Waterman. 1984. In search of excellence: Lessons from America’s best-run companies. New York, NY: Warner Books.

The Policy Imperative

21

Peterson, M.A. 1995. How health policy information is used in Congress. In T.E. Mann and N.J. Ornstein, eds., Intensive care: How Congress shapes health policy. Washington, DC: American Enterprise Institute. Pineault, R., et al. 2005. Collectif de recherche sur l’organisation des services de première ligne au Québec: Rapport synthèse. Direction de la santé publique, Agence de développement de réseaux locaux de services de santé et de services sociaux de Montréal. Montréal. – 2007. Involving decision-makers in producing research Synthesis: The case of the research collective on primary healthcare in Quebec. Healthcare Policy 2(4): 1–17. – 2008. L’accessibilité et la continuité des services de santé: Une étude sur la première ligne au Québec. Final Report, Institut national de santé publique. www.inspq.qc.ca/publications/. – 2010. Conceptual and methodological challenges in producing research syntheses for decision and policy making: An illustrative case in primary healthcare. Evaluation 16(2): 137–52. Pressman, J., and A. Wildavsky. 1973. Implementation. Berkeley: University of California Press. Rochon, J. 1988. Commission d’enquête sur les services de santé et les services sociaux: Report. Les Publications du Québec, Government of Quebec. Romanov, R.J. 2002. Building on values: The future of health care in Canada, final report. Ottawa: Commission on the Future of Health Care in Canada, Government of Canada. Sabatier, P.A. 1993. Policy change over a decade or more. In P.A. Sabatier and H.C. Jenkins-Smith, eds., Policy change and learning: An advocacy coalition approach. Boulder, CO: Westview Press. Sansfaçon, J. 2009. CHUM – C’est à l’État de payer, editorial. Le Devoir, 9 March. Scharmer, O.C. 2007. Theory u. Cambridge: The Society for Organizational Learning. Sheldon, T.A. 2005. Making evidence synthesis more useful for management and policy-making. Journal of Health Services Research. 10(S1): 1–5. Sinclair, D. 2000. Looking back, looking forward: Seven-point action plan, working report. Toronto: Ontario Health Services/Care Restructuring Commission, Government of Ontario. Sullivan, T., et al. 2008. Améliorer la responsabilité clinique et la performance en cancérologie. Pratique et organisation des soins. 39(3): 207–15.

22

Why extra?

The Commonwealth Fund. 2008. Why not the best? Results from the national scoreboard on U.S. health system performance. Commission on High Performance Health System. Tuohy, C.H. 1999. Accidental logics: The dynamics of change in the health care arena in the United States, Britain and Canada. New York, NY: Oxford University Press. Vadeboncoeur, A., and M. Goulet. 2009. Au nom de Médecins québécois pour le régime public, CHUM: Arrêtez le PPP, opinion. Le Devoir. 6 March.

2 The Management Imperatives for Leading Clinical Change robert s . bell

Peter Drucker is said to have described the modern-day hospital as the most complex organizational entity ever created (Seeman and Brown 2006). The high volume, complexity, and potential risks of treatment provided within a setting of constrained time, space, and financial resources, coupled with an unusual human resource model that provides physicians with “privileges” to use hospital resources, all serve to create a challenging and potentially high-risk environment. The rapid evolution of medical science and technology, along with the demand for more expensive treatment and the limitation of financial resources to pay for “best care,” create challenges for management staff. Furthermore, the public, in addition to increasing its demands for new generations of treatment, has become acutely aware that hospitals can be dangerous places. Effective health system performance demands a successful integration of medical and management science. This ambiguous relationship creates a crucial tension in the acute care sector; it requires an educational focus to address it, and it requires a leader who can administer both sciences well. However, these two areas of expertise are rarely found in the same individual. The Institute of Medicine report (Kohn et al. 2000) and similar descriptions of hospital complications and mortality have stimulated an increased emphasis on improving hospital quality and safety. Health care managers recognize that the rapid pace of change in costeffective and safe best practices means that system leadership requires constant evolution of clinical care based on an interpretation of evidence. Leading change in health care requires an understanding of

24

Why extra?

two different aspects of evidence-based practice: evidence-based medicine and evidence-based management (Guyatt et al. 1992; Sackett et al. 1996; Gray 2001; Golden 2006). The interaction between these two principles is sometimes complementary and sometimes contradictory, but both need to be understood to provide effective health care leadership. Clinicians around the world have developed a strong affinity for evidence-based medicine, which was introduced in Canada (and internationally) by Sackett and colleagues at McMaster University (Guyatt et al. 1992; Sackett et al. 1996). The potential advantages of evidence-based medicine were initially described by Scottish epidemiologist Archie Cochrane in the 1970s. His work has been recognized by the naming of centres of evidence-based medical research – Cochrane Centres – and by an online resource, the Cochrane Collaboration, which codifies “best evidence” for a variety of medical conditions (Pfeffer and Sutton 2006). Indeed, not only is understanding the use of evidence crucial for understanding the best treatment for a patient, but the use of evidence is also fundamental in providing leadership to the health care system. It is necessary for health care leaders to understand evidence, to use evidence in a compelling manner to make the case for change within the health care system, to design change initiatives based on reliable information in the literature, and then to develop evaluation processes that analyze whether change is effective. In creating organizational change in the health care sector, physician groups are often the most resistant; in this instance, the effective use of information and evidence can be an enabler. In designing courses for the health care leader of the future, teaching the use of evidence to design future models of care and to engage physicians will be essential. Despite the need for managers to understand it, evidence-based medicine is frequently used as a “showstopper” or roadblock in avoiding clinical change that is crucial for improving the efficiency or quality of care. As will be described below, Computerized Physician Order Entry (Tamblyn et al. 2003) was initially resisted at the University Health Network because there was no Level 1 evidence that would support its adoption. Managers need to anticipate this resistance and use the techniques of evidence-based management to counter these showstoppers through artful use of evidence and small tests of change.

The Management Imperatives

25

In this chapter I will use Kotter’s eight-step model for effective change management (Kotter and Cohen 2002) to describe how evidence-based medicine and evidence-based management are both essential for effective health care leadership (Guyatt et al. 1992; Sackett et al. 1996; Golden 2006). The appreciation of evidence-based medicine is essential in forming a coalition for change, creating a vision for change, and removing obstacles to change. On the other hand, evidence-based management is crucial in creating short-term wins and especially in building and sustaining change. However, before describing how both evidence-based medicine and evidence-based management can be used in successfully leading clinical change, it is necessary to describe what each of the terms means.

Evidence-Based Medicine The term “evidence-based medicine” was first used in the literature by Guyatt et al.(1992). Using statistical techniques including metaanalysis of the medical literature, risk-benefit analysis, and randomized clinical trials, evidence-based medicine attempts to educate clinicians to make use of the best evidence available in the medical literature in their daily care of patients. Evidence-based medicine has three major themes. The first is to interpret the literature in understanding the best treatment to apply to a group of patients with a particular condition and to use this treatment philosophy consistently. The second theme emphasizes systematic review of medical literature to evaluate the best studies on specific topics. This process can be undertaken informally (for example, in a journal club), or it may use computerized data-mining technology. Indeed, the use of information technology may represent the only rational method for turning large volumes of medical studies into balanced, reliable advice for clinicians. Finally, evidencebased medicine can be understood as a medical “movement” in which advocates work to popularize the method and the usefulness of the practice to the public, patient communities, and educational institutions. Managers and leaders need to be aware of this movement in collaborating with physicians in developing change initiatives within the hospital. Clinicians generally have a high degree of respect for evidence-based medicine, and managers can use this high regard in recruiting clinicians to support change initiatives. Alternatively, the absence of respect for evidence-based medicine in

26

Why extra?

developing change initiatives can encourage clinicians to sabotage attempts to improve the quality of care. Evidence-based guidelines represent the practice of evidence-based medicine at the health system level. This includes producing guidelines, policies, and decisions for funding new drugs or medical interventions. Improving quality within a hospital or health care system often relies on the implementation of standardized care, such as standardized “bundles” of care protocols advocated by the Institute for Healthcare Improvement and Safer Healthcare Now! Managers can use clinicians’ respect for evidence-based management and evidence-based guidelines to create change initiatives that emphasize standardization of care to improve quality. Evidence-based medicine is used in reviewing the medical literature to determine what evidence can be relied on in developing best practices and what investigations and treatments should be used with an individual patient. Within the health system, evidence-based guidelines are useful in establishing standardized protocols for enhancing clinical quality and eliminating errors and omissions in treating standard conditions. Evidence-based medicine and evidence-based guidelines are also essential in deciding what investigations and treatments should be funded within a public health system or what services should be covered by third-party insurers. There is no question that understanding the principles of evidence-based medicine is necessary for effective leadership of clinical change.

Evidence-Based Management Evidence-based management is an emerging movement to explicitly use the current best evidence in management decision making. Evidencebased management makes managerial decisions and creates organizational practices informed by the best available scientific evidence. Unlike their counterparts in medicine, the judgments developed by evidencebased management also consider circumstances and ethical concerns. Contemporary managers and management educators make limited use of the behavioral-science evidence base relevant to effective management practice, but this practice has special relevance to the field of health care management and leadership, since so many clinicians are well trained in medicine rather than in management science. Efforts to promote evidence-based management in the business community face greater challenges than in health care. Unlike medicine,

The Management Imperatives

27

nursing, and education, health care management is not a profession. Managers as a group have no shared knowledge characteristics, making it unlikely that peer pressure will be exerted to promote the use of evidence by any manager who refuses to do so. Managerial success may be achieved using a diversity of techniques and tools. Evidencebased management techniques will not guarantee success for a manager unless they are complemented by the use of other management styles and tools. Relatively little shared terminology exists for evidence-based management, making it difficult for managers to hold discussions of evidence or evidence-based practices. For this reason, the adoption of evidence-based practices is likely to be organization-specific, with leaders making a conscious decision to build an evidence-based culture (Pfeffer and Sutton 2006). Practices in an evidence-based organizational culture include the systematic accumulation and analysis of data gathered on the organization and its functioning, the success or failure of problem-based interventions and the discussion of research summaries by managers and staff, and making decisions informed by the best available research and organizational information. Organizations successfully pursuing evidence-based management typically go through many cycles of experimentation and redesign of their practices to create an evidence-based culture consistent with their values and mission. Indeed, one of the major differences between evidence-based medicine and evidence-based management is that while evidence-based medicine interprets scientific data derived from the medical literature in order to formulate a best practice, the management literature does not offer the same richness of description of what works and what does not work within a business enterprise, for two reasons. First, business initiatives are rarely studied with the rigour of the scientific observations used in medical investigations. Corporations are interested in only one outcome when introducing a new business initiative: the impact of the initiative on shareholder value. Second, if the initiative relates to a new business process or technique, it is likely that the corporation will carefully protect the intervention as a proprietary business secret, rather than freely describing the initiative. The absence of the medical scientific literature in evidence-based management means that managers generally need to develop their own hypotheses, data bases, and data interpretation to understand how to develop meaningful change initiatives. The sequential development of

28

Why extra?

interventions, data collection, and data interpretation is identical to the virtuous cycle of quality improvement described by Deming and adopted by quality practitioners around the world, namely, Plan, Do, Study, Act. In leading clinical change, evidence-based management follows a three-step process that can be aligned with Kotter’s eight-step change model. 1 What are we trying to accomplish? This question is intended to help the manager be clear about the improvements that he or she would like to make and about what results he or she would like to achieve. Having a clear vision is crucial. 2 How will we know that a change is an improvement? Without measurement, it is impossible to know whether things have improved. It is crucial to decide how things will be different when change is implemented and to agree about what data are required to measure the impact of change. Typical change initiatives will measure, for example, whether a patient outcome improves or whether a process is offered in a more cost-effective fashion. 3 What initiatives will likely lead to an improvement? It is necessary to decide what initiatives or interventions will be used to achieve quality improvement. What evidence exists elsewhere about what is most likely to work? What does the team think is a good idea? What have other people done that could be attempted? Once an initial plan is agreed on, evidence-based management calls for the implementation of small tests. Managers can use plando-study-act cycles to test out the interventions or initiatives developed from the third question, What changes can we make that will lead to an improvement? The key to plan-do-study-act cycles is to try out change initiatives on a small scale to begin with and to rely on using many consecutive cycles to build up information about how effective the intervention is. This makes it easier to get started, gives results rapidly, and reduces the risk of something going wrong and having a major negative impact. If what is tried does not work as well as hoped, it is easy for managers to go back to the way they did things before. When enough information is known for managers to feel confident about the change initiative, it can then be implemented as part of the routine system.

The Management Imperatives

29

Think of a “small” plan-do-study-act cycle in terms of the scope of your test. You might, for example, like to “plan” your cycle over one day, with one person or in one clinic. You might wish to “do” the change with the last ten patients seen, the last twenty referrals made, or the next dozen reports. The “study” part of the cycle gives you the opportunity to reflect on what happened, think about what you have learned, and build your knowledge for further improvement. Finally, you can move on to your next steps – the “act” part of the cycle. Do you need to run the same cycle again, gathering more evidence or making some modifications based on what you have learned? Or do you need to develop further cycles to move your work forward? At present there are initiatives in several parts of the world, including in Canada through the Canadian Health Services Research Foundation (CHSRF), to begin building communities promoting evidence-based management. Another example is the Center for Health Management Research affiliated with the Health Research & Educational Trust of the American Hospital Association. It is notable that a significant fraction of the evidence-based management leaders are concentrated in the health care field.

Change Management In The Heart of Change Kotter and Cohen (2002) describe eight essential steps for successful organizational change: (1) building a sense of urgency, (2) creating a powerful coalition, (3) creating a vision for change, (4) communicating the vision, (5) removing obstacles, (6) creating short-term wins, (7) building on successful change, and (8) anchoring change in corporate culture. Many of these necessary steps can be facilitated or inhibited by evidencebased management. Knowledge of the risks and rewards offered by each methodology can help the health care manager in developing a successful change project. In describing the processes of change, I will use the real- life example of a major change management initiative at the University Health Network (UHN). In 2002, our board challenged management and clinical leaders at the UHN to implement Computerized Physician Order Entry (CPOE) (Tamblyn et al. 2003). This created a major change management challenge.

30

Why extra?

Developing Urgency In developing the burning platform for change, evidence-based medicine can be applied in determining whether best practices are currently used and whether outcomes are consistent with best practices. In searching for evidence that change is required, it is useful to search the literature for Level 1 evidence that some institutions have outcomes better than those found at a particular organization. There was no question in our CPOE challenge that medication errors contributed to in-hospital morbidity and sometimes mortality. However, there was no Level 1 evidence that CPOE could solve these errors, despite evidence that errors are reduced by CPOE. In the case of CPOE, evidence-based medicine was used as a potential “show-stopper” for doctors to consider, since there was no Level 1 evidence that CPOE improves patient outcome. However, we felt that the Institute of Medicine report of hospital risk could be used in developing a sense of urgency among the physicians (Kohn et al. 2000). This was crucial, since physician work flow would be radically altered by implementation of CPOE. Creating a Powerful Coalition In creating change management advocates in the hospital, physicians play a crucial role. However, they are often one of the more difficult groups to recruit. The reasons are obvious. Physicians are generally not paid by the hospital, and joining a quality team is either nonremunerative or less lucrative for them than working in their practice. Also, physicians tend to be traditionalists in maintaining their practice habits. Nevertheless, the use of evidence-based management techniques can help in recruiting doctors to the change team. Describing the problem and the change anticipated in terms that emphasize measurement and evidence will often engage physicians by appealing to their scientific appreciation for data. If the clinical change has actually been tested in a randomized clinical trial, then an evidence-based approach can be useful in recruiting physicians to the initiative. As mentioned, there was no Level 1 evidence that could be used in recruiting physicians to CPOE. However, support from the physician leadership was achieved based on the Institute of Medicine report and the recognition that our board was committed to reducing drug errors.

The Management Imperatives

31

Measuring drug errors at our hospital and recording the proportion that could be attributed to errors in interpreting hand-written orders provided useful data for developing a strong coalition. Creating a Vision for Change The vision for change should be based on values that clinicians appreciate and will work to improve. For example, describing a flow initiative using a vision that emphasizes budget reduction through bed closures will not appeal to many clinicians. Describing the same initiatives as important because the outcome will improve the patient experience is much more likely to be effective. In choosing the measurements and variables that will be analyzed in the change initiative, practitioners of evidence-based management should describe outcomes that will appeal to clinical aspirations rather than budgetary imperatives. In the case of CPOE, a vision of patient safety was relatively easy to use in recruiting physicians. In following the plan-do-study-act approach, it was also crucial that we used a pilot study in order to “work out snags” and to recruit physicians to a vision of change. Removing Obstacles Experienced practitioners of evidence-based management will recognize that roadblocks and obstacles will frequently be used by clinical staff to avoid participating in the change initiative. Staff may also try to derail the initiative through arch criticism. In fact, even principles of evidence-based medicine are frequently used by clinicians in subverting change initiatives; the usual commentary heard is that “there is no Level 1 evidence supporting the use of this initiative in solving this problem.” It is crucial that evidence-based management practitioners anticipate this predictable obstacle and remind clinicians that not all clinical problems can be treated using evidencebased medicine and that sometimes treatment must be attempted in the absence of Level 1 evidence. Again with the CPOE case, the use of inspirational goals of improving patient safety along with the willingness to “try it out” with a pilot study were key in reducing “showstoppers” and roadblocks to CPOE.

32

Why extra?

Creating Short-Term Wins The concept of short-term wins is synonymous with the concept of small tests of change popularized by lean-theory practitioners. The evidence-based management practitioner will organize a team to define what needs to be changed, how the change will be measured, and what initiatives may accomplish the required change. The process next dictates a rapid plan-do-study-act cycle that will test the effect of the initiative with a small test of change. The evidencebased medicine practitioner will frequently subvert this process by wanting to run a larger, more time-consuming, and more expensive management initiative that will definitively address the entire problem under scrutiny. For example, in attempting to study flow improvements, an evidence-based medicine practitioner may suggest several initiatives that will improve flow and also suggest that all should be implemented simultaneously on one unit using another unit as a control. In contrast, the evidence-based management practitioner will test one small change initiative by comparing it to the previous performance of the unit over a short period (serial cohort study) rather than comparing it to another unit (randomized controlled trial design). In CPOE we started a small pilot project on one unit and quickly realized that our software was slow and inadequate and would not be accepted by physicians. It was crucial that we started with a small test of change with a small group of committed physicians. Had we started a Phase III study over several clinical units, we might have destroyed any momentum for CPOE. With this initial small “test of change,” we made changes to the software and were able to demonstrate how effective the change might be. Building on Successful Change The evidence-based management practitioner is delighted to build change using successive plan-do-study-act cycles, adding on initiatives that improve outcomes and rapidly discarding initiatives that do not achieve the desired impact. The devotee recognizes that Level 1 evidence requires that variables should be kept as constant as possible so that the impact of any single initiative can be understood. The evidence-based management practitioner knows that change management is attempting to modify human and organizational

The Management Imperatives

33

behaviour, rather than demonstrating an impact on pathobiology. In order to understand whether an organizational initiative is effective, it is useful to assess its utility with several different outcome measures, some of which may be qualitative. With our CPOE project, we built unit upon unit, making software revisions and implementation changes as we learned from each unit. At the same time we measured the reduction in medication orders floor by floor, building support for the change by showing how we were improving patient safety. Anchoring Change in Corporate Culture Evidence-based management emphasizes involving teams in problem analysis, developing initiatives, and testing initiatives through the plan-do-study-act process. The value of team engagement is in large part related to skill development and empowerment within the workforce, in addition to the improvement created by the change initiative. Within the hospital, evidence-based management initiatives encourage professional and non-professional staff to contribute to change management and critically evaluate and improve their work processes. The analytical skills that staff members gain by applying evidence-based management result in a corporate culture that values and contributes to clinical improvement strategies. In our hospital, the CPOE change has now been anchored in a culture of patient safety. Once a major change in work processes has been accomplished and once data have shown that the change is effective, the change should be used in facilitating further quality improvements.

Assessment of Statistical Variation and Secondary Consequences in Evidence-Based Management As mentioned above, practitioners of evidence-based medicine frequently find evidence-based management “messy,” since variables cannot be as neatly controlled or randomly distributed in a management initiative as in a randomized controlled trial. Evidence-based management focuses on rapid tests of change, and repeatable plando-study-act cycles of change create a more nimble environment for quality improvement in the hospital setting. However, despite the utility of evidence-based management in creating quality improvement

34

Why extra?

processes, the skilled manager should be aware of the perils of inadequate sample size and random variation in assessing whether an initiative has indeed enhanced quality within a clinical enterprise. Evidence-based medicine practitioners recognize that testing the success of an intervention requires a rigorous statistical analysis of whether real change has occurred. This testing has two elements: analysing whether the observed change is clinically significant (whether the change has sufficient clinical impact to warrant continuing the change initiative and analysing whether the observed change is statistically significant (whether the change is related to the initiative or possibly observed through random variation in the variable). The answer to these questions requires using statistical estimates before starting the test and statistical testing after collecting the data. Evidence-based management practitioners rarely subject their quality initiatives to rigorous statistical testing. Results of initiatives are reviewed for their effect by analyzing “trend lines” in the variable of interest rather than testing the outcome at one time for evidence of impact. For example, hospitals frequently designate improvement in mortality as one of their major quality improvement initiatives. Risk-adjusted mortality has been popularized as an indicator of hospital safety and several initiatives have been proposed to improve mortality (for example, Safer Healthcare Now! initiatives). A hospital might select one quality initiative, such as implementation of critical care response teams, to try to improve mortality results. Testing the implementation of this initiative would require rigorous statistical modelling, including determining the extent of mortality reduction that would be clinically significant and sample size determination of the number of patients who needed to be at risk for mortality before and after the change was made. This rigorous interpretation of the impact of response teams was undertaken in one study completed in Australia but failed to show a significant impact on reduction in mortality, probably because the sample size was not sufficiently large (Hillman et al. 2005). Despite the lack of a demonstrated impact on reducing mortality, critical care response teams have been adopted as a quality initiative by many hospitals because of secondary consequences. First of all, many hospitals have shown that the implementation of these teams does result in a significant reduction in cardio-respiratory arrests in the hospital units. Second, the presence of critical care staff consulting

The Management Imperatives

35

on the units is a terrific learning opportunity for health care staff with limited critical care skills. These secondary consequences may be positive or may occasionally be negative. A negative unintended consequence of evidencebased management improvements can be seen with various flow initiatives targeted at reducing waiting times in the hospital emergency department. Effective management of patients requiring admission from the emergency department requires that patients are admitted to any available bed, including surgical beds. The unintended consequence of this “surge admission protocol” is that surgical beds may be used by non-surgical patients, reducing surgical capacity with a consequent lengthening of surgical wait times. These potential unintended consequences mean that evidence-based management practitioners should measure a variety of indicators in addition to the primary measure of interest. We have seen that evidence-based management suffers from avoiding (in most cases) a rigorous approach to statistical evaluation of the impact of initiatives on outcomes. This potential failing means that it is important for its practitioners to use many different measures with different contexts in evaluating the success of their initiatives. In addition to the primary measure of interest (standardized mortality rates or wait times for admission through the emergency department), it is wise to employ different measures in assessing the impact of initiatives; for example, focus groups of ward nurses who have encountered the critical care response teams, measurement of satisfaction in patients admitted through the emergency department, or surgical cancellation rates for initiatives designed to reduce emergency department waits.

Summary Knowledge and understanding of both evidence-based medicine and evidence-based management are useful for improving the quality of care and leading change in the hospital. Evidence-based medicine is crucial in implementing best-practice guidelines and in standardizing practice based on Level 1 evidence. On the other hand, Level 1 evidence is not available to direct many of the quality issues that health care leaders target for improvement. When Level 1 evidence is not available, evidence-based management is useful for analyzing the problem and directing the rapid-cycle improvement process.

36

Why extra?

Leaders of the Canadian health system recognize that engaging our managers and executives in understanding how evidence should be used in understanding a management problem, developing the coalition to tackle the problem, analyzing initiatives that may address the problem, and assessing the success of the change are all absolutely necessary in creating a sustainable and quality-driven health system. Courses such as the EXTRA/FORCES program contribute to a leader’s ability to use evidence to create change and will be increasingly important in our complex industry.

References Golden, B. 2006. Transforming Healthcare Organizations. Healthcare Quarterly 10(Sp): 10–9. Gray, J. 2001. Evidence-based healthcare: How to make health policy and management decisions. Edinburgh: Churchill Livingstone. Guyatt, G., J. Cairns, D. Churchill, et al. 1992. Evidence-based medicine: A new approach to teaching the practice of medicine. Journal of the American Medical Association 268(17): 2420–5. Hillman, K., J. Chen, M. Cretikos, et al. 2005. Introduction of the medical emergency team (MET) system: A cluster-randomised controlled trial. Lancet 365(9477): 2091–7. Kohn L., J. Corrigan, and M. Donaldson. 2000. To err is human: Building a safer health system. Committee on Quality of Health Care in America, Institute of Medicine. Kotter, J.P., and D.S. Cohen. 2002. The heart of change: Real life stories of how people changed their organizations. Cambridge: HBS Press. Pfeffer J., and R. Sutton. 2006. Hard facts, dangerous half-truths and total nonsense: Profiting from evidence-based management. Cambridge: Harvard Business School Press. Sackett, D.L., W.M. Rosenberg, J.A. Gray, R.B. Haynes, and W.S. Richardson. 1996. Evidence based medicine: What it is and what it isn’t. British Medical Journal. 312(7023): 71–2. Seeman N., and A. Brown. 2006. Remembering Peter Drucker: Inspiring the quality revolution in healthcare. Healthcare Quarterly 9(1): 50–4. Tamblyn, R., et al. 2003. The medical office of the 21st century: Effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. Canadian Medical Association Journal. 169(6): 549–56.

section two

The Curriculum Elements

Section 2 opens with a short overview in chapter 3 of the EXTRA program’s overall approach to the curriculum. It highlights both the core content and the enabling elements of health information access and mentoring. The next two chapters, by Lavis and by Streiner and colleagues, cover the foundational curriculum material of the EXTRA program. Chapter 4 looks at various strategies to incorporate research-based evidence within the decision-making process. An approach is proposed that systematizes the use of evidence while taking into account organizational context. Particular attention is paid to strategies that create more effective partnership between researchers and decision makers. Lavis highlights the challenges faced by practitioners as they try to create an evidenceinformed agenda in their organizations. His curriculum for this work has evolved and now focuses on defining the problem, choosing among the options, pulling it together, and creating a sustained basis for the use of research. Of particular focus in this first instalment are various sources and approaches to finding evidence for action-oriented managers and policy-makers. In chapter 5 Streiner and colleagues look at the types of research that may inform decision making in health care. The material is tailored to improve research literacy among the health executive fellows, who bring a broad range of prior exposures, from none to active researcher. The material explores issues in the presentation of research data and in quantitative and qualitative research. Streiner and colleagues use real-world examples, incorporate health economics research, illustrate how to exploit administrative data, and

38

The Curriculum Elements

offer tips on how to evaluate the quality of research work. This approach helps fellows to triangulate evidence, using different sources and methods, in a way that respects the complex problem-solving needs of a health care management organization. Chapter 6, by Sullivan et al., focuses on leadership and highlights several key attributes and competencies for health care leaders interested in advancing evidence. The chapter focuses on three broad leadership competencies: strategy development, people management, and execution. A short list of competencies highlights areas for individual skill development that can equip fellows to advance evidence use at the local-program, institutional, and broader sociopolitical levels. In addition, fellows are taught the key points of strategic leverage that leaders can use to motivate organizational change in favour of better quality and performance. Chapter 7, by Langley et al., deals with the implementation and the sustainability of evidence-informed change in health care organizations. The chapter uses two case studies to focus not on strategy but, rather, on the role of evidence in the implementation of change. The interface between research-based evidence or rational information and the politics of decision making is explored. As noted in chapter 1, evidence may bring all sorts of effects that do not necessarily culminate in so-called rational decisions. In chapter 8 Sheps introduces one of the “horizontal enabling features” of the curriculum, namely, mentoring. He reviews the mentoring literature before detailing the unique model adopted for the EXTRA program, contrasting this resource- intensive model with the traditional approach prescribed in the mentoring literature. In addition, he illustrates how the EXTRA program’s mentoring model has evolved organically, based in large part on feedback from fellows (one of the key changes was the development of a regional structure) and provides some feedback on the mentoring experience through the voices of fellows. The other horizontal enabling feature of the program – the health information management curriculum – is touched on in chapter 3. Chapter 9, by Anderson and colleagues, highlights the evaluation efforts in the first five cohorts of EXTRA. The authors assess the utility of the program from the fellows’ perspectives, based on feedback from individual surveys and focus groups. The program itself is evaluation-intensive, with module evaluations conducted during every teaching episode and following each cohort. This approach

The Curriculum Elements

39

to ongoing evaluation over the last five years has helped the EXTRA curriculum to evolve to better meet fellows’ needs. To date, evaluations have focused on the individual fellows, their acquisition of new knowledge, and their appreciation of the program elements and the curriculum format. Currently and in future, greater attention will be placed on assessing the organizational impacts of the program as they become more evident through the accumulation of multiple fellows in single organizations, particularly with teambased applications of multiple fellows from single institutions.

This page intentionally left blank

3 extra

Progam Basics

terrence sullivan , jean - louis denis , and samuel b . sheps

The promotion of the EXTRA program, a pan-Canadian, fully bilingual program, is supported by strategic advice and the annual selection of fellows under the aegis of a high-profile advisory council that includes a former provincial minister of health as its chair. A detailed overview of the program and the curriculum appeared in Denis et al. (2008). Program aims are defined at the three levels of professional development, organizational performance, and, ultimately, health system benefits: •

•

Professional (individual) development. EXTRA aims to improve the competency of health care executives to understand, acquire, assess, and use research evidence in the management of health care organizations. It also encourages greater teamwork in management between physicians, nurses, and other executives by explicitly recruiting them to the program in approximately equal numbers and encouraging collaborative teamwork with a variety of exercises during the program. Improving organizational performance. The program aims to strengthen an evidence-informed decision-making culture at each of the participant’s home institutions to drive performance improvements. It does this in various ways, including a requirement for sign-off on the fellowship and a change project by the home institution’s chief executive, attendance by the chief executive at the final residency session and joint presentation with the fellow, and development of a chief executives’ forum for each

42

•

The Curriculum Elements

year of graduating fellows. The admission criteria were recently expanded (2007) to allow organizations to participate by sending interdisciplinary team applications for two to three senior executives focussed on a large change initiative. In 2008 a limited number of fellows from governmental policy environments were admitted to the program. Health system benefits. It is expected that these program initiatives will increase the overall organizational impact of training on the use of evidence in decision making, driving organizational improvement and ultimately system-level benefits.

Program Philosophy As LaMarche et al. and Bell note in the opening chapters of this book, numerous challenges are associated with the importation of evidence into executive-level decision making in health care organizations. Taking into account these challenges, the EXTRA program curriculum is structured around four main pillars: residency sessions where selected fellows develop individual capabilities to practice and promote evidence-informed management in health care settings; the development and implementation of an intervention project through which fellows apply problem-definition and problem-solving approaches in their own organizational contexts; an information management component across the whole curriculum, including a web-based technology (“the desktop”) to support fellows’ learning within and outside residency sessions; and a mentoring system to coach fellows in the development and implementation of their home institution intervention projects. Fellows are supported by learning networks and a community of practice after the completion of the program to maintain their skills and spread their evidence-informed management knowledge to colleagues inside and outside their own organizations. These program components are based on a broad conception of research evidence to support decision making in health care settings (Culyer and Lomas 2006). This broad view encompases the effectiveness of organizational and managerial interventions (what works?) and the likely effectiveness of organizational change processes in the context in which such interventions will be introduced (will it work here?). EXTRA is designed to integrate these two broad

Program Basics

43

categories of evidence as the basis for improved management in health care systems and organizations.

Program Content EXTRA fellows participate in a two-week summer and a one-week winter residency module over each of two years. The curriculum modules are planned to follow a logical sequence, from an understanding of research and evidence to its application in organizational change. Each module is delivered by experienced faculty using a variety of adult learning approaches, including lectures, case studies, and paired exercises. While fellows gain a solid grounding in theory, their focus is on the application of theory in the context of their intervention project. Residency sessions are the cornerstone of a comprehensive learning design that includes IT support, mentors, academic advisors, between-session exercises, and an intervention project. Chapters 4–8 track the initial modules of the curriculum, with the exception of the sixth module (the synthesis seminar), which is dedicated to the presentation of the intervention projects by fellows in partnership with his, her, or their chief executive. The program components reflect an executive/adult pedagogical approach to learning; the views of the fellows are made clear in the third part of this volume. The program emphasizes the formative rather than the summative evaluation of fellows’ performance within the program. At the end of their program (module 6), fellows present and receive feedback on their intervention projects in front of a jury composed of top-level executives and policy-makers, and in collaboration with their chief executives. In addition to the mentoring program noted in the introduction, a second horizontal curriculum element woven through the modules has been health information management (HIM). This part of the curriculum includes the use of online resources and an integrated program desktop to support the entire program, along with its curriculum and the presentation of online resources. The curriculum and the desktop are supported by colleagues at the Centre for Health Evidence at the University of Alberta (http://www.cche.net/). At the heart of both the HIM curriculum and the desktop is the recognition that health information literacy is now critical to evidence-informed

44

The Curriculum Elements

health services practice and management (Hayward et al. 2006). Health care management increasingly depends on communication and information systems and technologies, as well as on the Web. Managers need familiarity with the best information management tools to acquire the knowledge and the technical skills to make optimal decisions and drive performance improvement in patient services. As fellows move through the EXTRA program, they are exposed to new ways of working with information. Recognizing this, the HIM curriculum teaches skills in a just-in-time fashion, at the point where they can be applied. It is spread across the six EXTRA modules and is taught using a combination of face-to-face, self-directed, and group learning. The curriculum is divided into five general groups of topics: • • • • •

information communication technologies; privacy, confidentiality, and security; personal-information management; evidence information management; and organizational-information management.

The EXTRA desktop provides the learning content and resources for these topics and also acts as the laboratory in which fellows practice and further develop their skills. Indeed, fellows also have access to the desktop and its resources once they have graduated. As the cohorts move through the EXTRA program, the desktop becomes an important social networking tool that supports an emerging series of cohorts interested in further contact and support. A formal community-of-practice area is now part of the desktop managed by a group of EXTRA graduates.

Intervention Projects The key task that the fellows must complete during the fellowship is the Intervention Project (IP). It provides fellows with the opportunity to define and resolve a key problem within their own organization. The IPs have two main objectives: allowing fellows to practice evidence-informed management and developing organizations’ capacities to achieve evidence-informed change. The IPs are closely linked to the contents of the modules in that the residency sessions make use of fellows’ IPs as cases or illustrations.

Program Basics

45

To ensure that the fellows have support in realising these IPs, a mentoring system has been set up (see chapter 8) that operates during and between residency sessions. Both the mentors and the fellows operate under a detailed guidance document for the development of the intervention projects. The mentoring system is essential to ensure proper follow-up and guidance throughout the program, and fellows receive formal feedback through progress reports on their intervention projects from faculty associated with the regional mentoring centres. Overall, four major outcomes are expected from the work of the fellows in the program: fellows acquire new skill sets and knowledge from participating in the program; fellows improve their capacity to collaborate as evidence-informed decision makers between professional streams; fellows apply the skills learned and use research evidence to bring about organizational change; the skills needed for improved use of research in management are spread beyond those enrolled as fellows in EXTRA. As of the middle of 2010, 7 cohorts totalling 174 fellows had entered the program. Chapter 9 provides a richer characterization of the individual fellows in each cohort, their regional presence, and their backgrounds. As noted in the introduction, an organizational-impact evaluation has just been finalized and is in preparation for separate publication. Many health care system initiatives are under way to develop exchanges between practising managers, policy-makers, and researchers (the Service Delivery and Organisation R&D program in the United Kingdom is one example, and the various “listening” exercises in Canada and the United Kingdom represent another). Natural companions for these initiatives are training programs like EXTRA that develop attitudes, skills, and competencies to drive evidence-informed improvements in quality and performance in our delivery systems.

References Black, N., and N. Mays. 1996. What is “development”? J Health Serv Res Policy 1:183–4. Culyer A., and J. Lomas. 2006. Deliberative processes and evidenceinformed decision making in health care: Do they work and how might we know? Evidence and Policy 2:357–71. Denis, J.-L., J. Lomas, and N. Stipich. 2008. Creating receptor capacity for research in the health system: The Executive Training for Research Application (EXTRA) program in Canada. jhsrp 13(1): 1–7.

46

The Curriculum Elements

Hayward, R.S., M. El-Hajj, T.K. Voth, and K. Deis. 2006. Patterns of use of decision support tools by clinicians. 1: amia Annu Symp Proc.: 329–33. Kovner, A.R., and T.G. Rundall. 2006. Evidence-based management reconsidered. Front Health Serv Manage 22:3–22.

4 Supporting Research Use in Policy and Management john n . lavis

What would you do if you found yourself in one of the following scenarios? •

•

•

•

You open your morning newspaper to find that the issue of doctor shortages has captured the front page again, and you know that as minister of health you will be asked by reporters later that day why this problem persists despite years of government activity. You hear from your chief nursing officer that a delegation of local physicians has demanded in a public forum that your hospital adopt a policy of “pay for performance” for its nurses, which they have argued has been “shown to work” everywhere it has been tried. You have been asked by the CEO of your regional health authority to develop an implementation plan for a new program targeted at diabetic patients and the solo physicians and multi-disciplinary health care teams that provide their care. Your assistant deputy minister has asked for a briefing about what the provincial ministry of health can do to respond to criticisms from a coalition of health-professional groups that the ministry uses a double standard – one for health professionals, who are expected to base their professional decisions on the best available research evidence, and another for the ministry itself, which never invokes research evidence in communicating the rationale for its decisions.

In the past you or your staff might have “muddled through” as best you or they could, drawing on the results of a quick Google

48

The Curriculum Elements

search, the insights from a brief chat with a colleague, or the advice from an expert whom you or a staff member know socially and sometimes call on professionally. But EXTRA’s module 1 was designed and is being continually adapted to assist health system managers and policy-makers to see why these approaches can sometimes lead to harmful or costly mistakes and how more and more resources and tools are becoming available to help them find and use research evidence efficiently as one input into addressing the pressing health challenges they face. The module does not purport to “do it all.” Instead it strives to break down a complex task into manageable steps, provide guidance about where to look for what and what to do with what is found, encourage both a systematic approach and transparent reporting about what was done, and reassure managers and policy-makers that they and their staff can do this as part of doing their job well, not as an “add-on.” As one senior civil servant said, “thanks for showing me that this isn’t rocket science.”

What Module 1 Seeks to Achieve and How Module 1 equips fellows, all of whom are health system managers and policy-makers, with the knowledge and skills to find and use research evidence efficiently in order to respond to scenarios like those described above. Scenario 1 requires, first and foremost, that senior civil servants decide whether the problem has been defined correctly, which is dealt with on day 1 (table 4.1). Scenario 2 requires that hospital staff determine whether pay for performance has been shown to achieve the desired effects under conditions like those faced locally and, if so, what if any harms (or unintended consequences) have been observed in other settings, what it would cost, and what, if anything, is known about its cost-effectiveness relative to other options, which is dealt with on day 2. Scenario 3 requires a mid-level staff person to identify likely barriers to implementation and what is known about the benefits, harms, and costs of strategies that would address these barriers, which is the focus of day 3. Day 4 provides fellows with an opportunity to learn from (and be inspired by) a major teaching hospital’s efforts to define a problem (namely, the lack of a “rhyme or reason” for why some technologies are paid for and others not), choose among options for addressing the problem, and implement the chosen option, as well as to share with other fellows how they have recast their “intervention project”

Supporting Research Use

49

in light of the data and research evidence available to define their problem, choose among options, and implement an option. Scenario 4 requires a policy analyst to draw on an existing framework or tool to assess the ministry’s capacity to find and use research evidence efficiently, which is dealt with on day 5, and to provide civil servants at a more senior level with ways to present areas of strength and opportunities for improvement for the assistant deputy minister to consider. A cross-cutting thematic focus on managing health information provides fellows with the particular knowledge and skills needed to manage their efforts to find and use research evidence. Problem definition involves identifying the underlying problem, indicators for establishing its magnitude and measuring progress in addressing it, comparisons for establishing whether it is getting better or worse over time or is smaller or larger than in other settings, and alternative ways of framing the issue that resonate with (and could motivate) different stakeholders. Is the underlying problem really a shortage of physicians or is it how they are distributed within a jurisdiction, or how they are paid (which allows them to practice in relatively over-served areas)? Knowing how to use “hedges” (i.e., validated search strategies) in MedLine can help busy health system managers and policy-makers to find administrative database studies and community surveys to assist with making comparisons, as well as to find qualitative studies to assist with understanding how different stakeholders view and experience a problem. Knowing how to define a problem using data and research evidence is often the biggest challenge for action-oriented managers and policymakers. They often have a solution in mind and find it hard to stand back and ask what problem it is that they are trying to solve. Choosing among options involves identifying viable options for addressing a problem, describing what is known (and not known) about each option’s benefits, harms, and costs (or cost-effectiveness), determining whether adaptations might be made to what has been tried in other settings and what is known about how these adaptations might alter the option’s benefits, harms and costs, and ascertaining the views and experiences of stakeholders and how they might influence the acceptability of an option or its benefits, harms, and costs. Has a policy of “pay for performance” for hospital nurses been found to be effective everywhere it has been tried and rigorously evaluated and, if so, were there any unintended consequences? Health system managers and policy-makers can answer such a

50

The Curriculum Elements

Table 4.1 General Structure of Module 1 Day 1 Defining the problem

Day 2

Day 3

Day 4

Day 5

Choosing among options

Implementing an option

Putting it all together

Supporting the use of research evidence

Managing health information

Note: The vertical arrows in the second row represent the core content of each day, which is delivered in both the mornings and the afternoons. The horizontal arrows in the fourth row represent complementary content on managing health information, which is delivered once in the middle of each day.

question if they know (1) how to find systematic reviews about health system arrangements in the Health Systems Evidence database (www.healthsystemsevidence.org); (2) how to distinguish among reviews that address benefits (reviews of effects), harms (reviews of observational studies), how and why options work (reviews of process evaluations), and stakeholders’ views and experiences (reviews of qualitative studies); (3) how to assess the quality and local applicability of any reviews they find; and (4) how to find and use economic evaluations in the NHS Economic Evaluation Database. Knowing how to assess options is second nature to most managers and policy-makers. What is more challenging is knowing how to match the right type of research evidence to the right part of their assessment and knowing how to assess what they find, particularly what it means for their setting. Implementing an option involves identifying barriers to implementation, which may be at the level of patients/citizens, providers, organizations, or systems, and then describing what’s known (and not known) about possible implementation strategies’ benefits, harms, and costs (or cost-effectiveness). How and why might diabetic patients, solo physicians, and multi-disciplinary teams resist a new program and what strategies might address these barriers? Knowing how to find and use systematic reviews of qualitative studies or observational studies can help managers and policy-makers to identify barriers, while knowing how to find and assess systematic reviews and economic evaluations as described in the preceding paragraph

Supporting Research Use

51

can help to describe what is known about implementation strategies. Much of what managers and policy-makers do involves trying to change the behaviours of patients and providers. A great deal of research evidence exists to assist them in doing so. Supporting the use of research evidence, on the other hand, requires a holistic assessment of what an organization or system does well, what it does poorly, and where strategic investments might yield the greatest returns. Is it the case that health system managers and policymakers have been holding health professionals to different standards? Or is it that until only very recently there weren’t the resources and tools that would have enabled managers and policy-makers to engage seriously with the research evidence about the problems they face, the options they’re considering, and the implementation plans they develop, and hence that would have enabled them to articulate whether and how research evidence informed these steps. Knowing that there are now frameworks available to assess whether organizations and systems have the right supports in place for this type of work and knowing that there are now tools to assess one particularly important type of support (the capacity to acquire, assess, adapt, and apply research evidence) can help managers and policy-makers to identify areas of strength and opportunities for improvement at the organizational and system level. Individuals, like fellows, can do only so much. The organizational and system have to do their part. Module 1 provides fellows with many opportunities to act on new-found knowledge and to hone newly acquired skills. Case examples of problem definition are provided in plenary sessions to elicit discussion. Exercises, such as the always difficult assessments of the local applicability of a systematic review, are conducted in small groups. Barriers to implementation are identified through brain-storming sessions. All the way along, fellows are applying what they’ve learned to their intervention project. Near the end of the week, all fellows assess their organization’s capacity to acquire, assess, adapt, and apply research evidence, and patterns in their assessments form the basis for large-group discussions on the final day. A successful module means that each fellow has mobilized data and research evidence to establish an evidence-informed approach to their intervention project (the problem they are addressing and the options and implementation plan they are considering) and to establish an approach to supporting the use of research evidence more generally in their organization or system.

52

The Curriculum Elements

How Module 1 Has Evolved and Why The description in the previous section makes it sound as if module 1 was all figured out from the beginning and has kept with a “tried and true” formula through its first five years. While some of the current module would be recognizable to the fellows in cohort 1, a great deal has changed over the years since they participated in the module. The changes range from the placement of module 1 within the overall curriculum to what is emphasized in the module (and how it is presented), to how the module reacts to the fellows and how the fellows react to the module, and to how fellows’ needs (and more generally health system managers’ and policy-makers’ needs) have spurred us and others to develop tailor-made resources for them. By the completion of the first residency session (i.e., the completion of original modules 1 and 2), a consensus had emerged that the order of the first two modules needed to be switched. Fellows had enjoyed sitting back and having research “demystified” for them in the original module 1, but within a day or two of module 2 beginning, their anxiety level was palpable. Suddenly they were expected to actually find and use research evidence to establish an evidenceinformed approach to their intervention project (the problem they were addressing and the options and implementation plan they were considering). And they had only a few days left to do so before facing alone the six months until the next residency session. So the original module 2 (on supporting the use of research evidence in management and policy-making, which is also the focus of this chapter) became module 1, and the original module 1 (on demystifying research evidence, which is the focus of the next chapter) became module 2. The change worked: anxiety levels were down among cohort 2 fellows, and these fellows got much more out of (newly renumbered) module 1 and got much more directly related to their intervention project out of (newly re-numbered) module 2. The relative emphasis in the module’s content and how it is presented have shifted over time. First, module 1 has grown increasingly pragmatic in its tone. Unconsciously or consciously the content was initially presented as an “add on” to managers and policy-makers already busy with a list of tasks, instead of it being presented as way to help them do their job better. Yes, Google, colleagues, and “expert” acquaintances can get fellows part way there. But they won’t get them through a meeting with a well-informed interest group or through an

Supporting Research Use

53

interview with a committed journalist. We learned to position assessments of the local applicability of a systematic review as an opportunity to come to terms with (and hence be able to defend in an articulate way) what could be learned from experiences in other settings. We no longer present it in what may have seemed in the early years like an academic exercise in critical appraisal. Second, module 1 has given greater attention to problem definition over time. The content about problems was initially quite weak, but over the years, as it became clear that all else hinged on the fellows getting this part of their intervention project right, the content was gradually strengthened and oriented to their needs. Third, module 1 has also given greater attention to systematic reviews over time. In large part this was spurred by the steady growth of systematic reviews addressing all facets of defining a problem, choosing among options, and implementing an option, as well as by the development of Health Systems Evidence, which made the search for relevant reviews less like trying to find a needle in a haystack. Fourth, module 1 has also given greater attention to data (not just research evidence) over time. This transition was spurred in part by the recognition that local data was critical to good problem definition and in part through interactions with faculty in other modules who were very focused on the use of performance data. Module 1 has reacted to the fellows just as the fellows have reacted to the module. Each new cohort brings new expectations. One year there was a “bumper crop” of fellows from public health, so relevant resources were identified and incorporated into the module. Another year teams became eligible to participate in EXTRA, so team time needed to be introduced in order to allow fellows both to apply their new-found knowledge and skills to their intervention project and to be exposed to new thinking in small groups comprised of fellows who were not on their team. Policy-makers have also recently been invited to participate in EXTRA, so the language and resources needed to be adjusted to meet their needs as well. The reverse impact has been more amusing. Over the years we began to notice that the fellows’ final presentations in module 6 had become “dry as dust.” They had begun to sound like researchers. We have now placed greater emphasis on the use of anecdotes in module 1 to ensure that the human-interest story that initially motivated the fellows (and will likely motivate others) does not get lost amidst the data and research evidence.

54

The Curriculum Elements

The final evolution of note relevant to module 1 involves how fellows’ needs (and more generally health system managers’ and policymakers’ needs) have spurred us and others to develop tailor-made resources for them. In the early years of EXTRA, module 1 made the best of resources produced for other target audiences. Over time we introduced progressively more fine-tuned reading notes to accompany these resources. We and others have now finally produced resources that are written specifically for health system managers and policy-makers (see the chsrf support tools in both English and French, referenced at the end of this chapter, and Lavis et al. 2009). Similarly, a self-assessment tool that was first introduced as an exercise requiring fellows to assess their organization’s capacity to find and use research evidence has now been turned into a resource that fellows can use to promote discussion, support change, and monitor improvements in their own organizations.

Where Do We Still Need to Improve? Module 1 remains a work in progress. For example, we continue to experiment with new ways to help fellows with their greatest challenge – problem definition. But the bigger challenge has become how to expand this capacity-building effort beyond the (roughly) twentyfour health system managers and policy-makers selected each year. A number of initiatives have emerged over the last few years, from Royal Roads University’s diploma program to the more informal sessions and workshops that we run for civil servants in federal and provincial ministries of health and in international agencies and for analysts and managers in stakeholder associations, both in Canada and elsewhere. But given the rapid turn-over of health system policymakers and the vast number of health system managers, we need a much broader array of formats, including 1.5 hours for civil servants at the assistant deputy minister level who set expectations for their staff and one day for policy analysts and advisors. And we need a much more institution-focused approach to capturing the benefits of these capacity-building efforts. Training 240 champions for and practitioners of evidence-informed management and policy-making is a huge success. But there’s much more work to be done. Evidence-informed health systems require a dynamic process of finding and using many types of research evidence in response to the emerging and perennial challenges faced in health systems. Strong

Supporting Research Use

55

leadership is required from above, and module 1 contributes in some small way to exposing the next generation of leaders to the knowledge and skills required to play such a leadership role. But evidence-informed health systems also require a dynamic continuing professional development program for staff and a dynamic resource- and tool-development program that ensures continued improvements in the resources and tools on which managers and policy-makers can draw. Here perhaps the most that module 1 can do is to sustain a laboratory for innovation in these domains, which is a role it has been playing to great effect in its first five years.

References CHSRF support tools. Retrieved from http:www.chsrf.ca/other_documents/ Support_tools_for_policy_makers/index)_e.php. Lavis, J.N., A.D. Oxman, S. Lewin, and A. Fretheim. 2009. Support tools for evidence-informed health policy making (STP). Introduction Health Research Policy and Systems. 7 (Suppl. 1): I1doi: 10.1186/1478–4505– 7–S1–I1.

5 Building Research Literacy for Management david l . streiner, paula n . goering , and jeffrey s . hoch

Providing Research Literacy Training to Health Executives Managers, by definition, manage. With rare exceptions, they do not do research themselves, and it is equally unlikely that they would turn to the primary literature to answer a specific question. More likely, they would ask someone on their staff to search the research evidence and provide them with the “bottom line.” Given that, it is legitimate to ask the question, Do managers need to know anything about statistics, research methods, policy analysis, or health economics? Managers may not be researchers, or even consumers of primary research studies, but for a number of reasons, they should be conversant in the language and methods of research. Paradoxically, one of the primary reasons for providing research training to managers is to remove the mystique that surrounds research. We are all familiar with television advertisements that claim that “research has shown ...” or, even more impressively, that “research at a major medical institution has proven …” and then proceed to peddle the latest advance in snake oil. A sage-looking person dressed in a pressed, white lab coat, with a stethoscope prominently in view, further lends credibility to the message. The assumption is that any assertion is more readily taken as “truth” when it is supported by “research.” However, as every researcher knows, some published studies truly belong in the New England Journal of Medicine or the Journal of the American Medical Association, while others are perhaps best suited to the Journal of Irreproducible Results.

Building Research Literacy

57

Not all research is created equal, and managers must learn to look upon research with a skeptical – albeit not a jaundiced – eye. A second rationale for providing research literacy training is to prepare managers for those occasions when they are asked, or forced by circumstances, to open or close hospital units. Often, these requests will be bolstered with articles demonstrating the need for a new program or with compelling arguments that money would be better spent if allocated to one service or intervention over another. The articles may be replete with terms such as p < .05, effect size, meta-analysis, or even more frighteningly, cost-benefit analysis, quality-adjusted life years, or marginal costs. It is also possible that the petitioners may use these terms during their presentations, accompanied by impressive charts, graphs, and tables. Managers must be in a position to understand what is being said and, equally importantly, to know how various ways of presenting data can end up clouding issues as much as illuminating them. Finally, even though managers may not be researchers, to some degree they must learn to think like them. This is especially true in times of constrained resources – a period that shows no signs of ending in the foreseeable future. Managers are accountable for the ways they spend their budgets. In order to justify a new or an existing program, it is no longer sufficient to say, “It seems to be working” or “The people who run the unit tell me it’s doing great things.” There is an increasing demand that programs be evaluated, and with more rigor than testimonials from patients or staff can hope to offer. In a similar vein, there is pressure from funders, patients, and staff for clinical interventions to be informed by evidence and for clinical service to be blended with research and education. These trends mean that managers must know how people from different backgrounds – administrators, clinicians, researchers and evaluators – perceive issues. Like the blind men feeling different parts of an elephant and coming to conflicting conclusions about the nature of the beast, people trained in various traditions see issues in health services delivery from varying perspectives. Being in the middle, the manager must learn to adopt different points of view and to some degree, see the world as researchers do. For these reasons, the research-literacy module has the title “Demystifying the Research World.” However, we always come back to the same premise that opened this chapter: managers will not become researchers. Nor, frankly, do we want them to. What we do

58

The Curriculum Elements

want is for them to understand the logic and conceptualization of statistics, research methods, and health economics: not the details. We also know that details, such as the difference between a t-test and a Mann-Whitney u test, or between cost-benefit and cost-utility analysis, will quickly be lost if they are not conducted on a daily or weekly basis. Since we do not expect these evaluations to be carried out this often (if ever), there is no sense in covering the minutia. However, a conceptual understanding of these topics, especially if the examples are grounded in experiences that the fellows have had, will endure. By providing resources that the managers can later refer to – introductory books, key readings, helpful websites, and the like – the fellows can fill in the details if and when they are needed.

Quantitative Methods The section of the “Demystifying the Research World” module has six units: two on statistics, three on research methods, and one on the critical evaluation of systematic reviews and meta-analyses. The first unit covers descriptive statistics – how to summarize data numerically and graphically. Although these may sound like technical details that are quickly forgotten, our belief is that managers will use – or at least run across – descriptive statistics on a regular basis. Managers must deal with facts and figures almost daily, whether they describe the average number of falls in a long-term care unit, admissions and discharges from programs, or requests from the community. Furthermore, managers are called on to provide statistical summaries of their activities to their superiors. Our experience, based on working in various health care organizations for over forty years, is that most summaries of data are done relatively poorly, using the wrong type of measure of a central tendency or graphs that fail to communicate the key message. We believe also that computers have made the problem worse, not better. Graphing programs that come bundled with software programs are chock-full of dazzling new features such as three-dimensional graphs, pie charts, stacked bar graphs, a variety of shapes for bars, and so on. The results are marvellous to the eye, but often completely misleading, especially when combined with commonly made mistakes such as y axes that do not start at zero (Norman and Streiner 2007). In the EXTRA training, fellows are not only taught how to present data; they are also told how not to present them.

Building Research Literacy

59

The second unit on statistics deals with the logic behind inferential statistics. Because it is highly doubtful that the fellows will ever be called upon to do a t-test or would even remember what one is, specific techniques are not mentioned at all. Rather, the module explains the logic behind that ubiquitous phrase “p < .05” and why dichotomizing findings into “proven” or “not proven” at this arbitrary value can lead to two types of errors: false positive and false negative results. The important point is that the conclusions of studies are probabilistic statements, and they may be wrong. The next three sessions deal with various aspects of research design. Even after five iterations, this section of the module is still a work in progress. Part of the difficulty in designing these sessions is a tension between two objectives, the first being simply to acquaint the fellows with terms they will encounter in their discussions with their staff, not to make researchers out of them. However, as part of their training, the fellows have another program objective: to implement a project in their organization and evaluate it, and evaluation requires some knowledge of research methods, both quantitative and qualitative. When the program first began, we placed a lot of emphasis on the randomized controlled trial (RCT), because it is generally considered to be the strongest design for determining the efficacy or effectiveness of clinical interventions (Haynes et al. 2006). Needless to say, it is still discussed, because much of the evidence that will be brought to managers in support of a proposed new program will consist of RCTs. However, we also realize that programs cannot always, or even often, be evaluated using this design: if a new unit is opened up in a hospital, it would be impossible to randomly admit patients for a number of years until it has proven its worth. Consequently, we have been devoting more time to quasi-experimental designs, such as interrupted time series, multiple base line studies, and so forth (Shadish et al. 2002; Streiner 1998). The second session in the research design component deals with the measurement of outcomes, primarily relative risk (RR) and relative benefit (RB). The rationale again is that much of the evidence presented to managers will use these indices to illustrate how a new and often very expensive drug or program will be “the greatest boon to humanity since physicians started washing their hands.” However, relative indices can hide a lot of mischief; if the proportion of people who benefit or who are at risk is very small, then an RR or RB of even 3.0 means that the intervention will be useless for the majority of

60

The Curriculum Elements

recipients (Streiner 2005). So it is necessary to supplement these figures with the number needed to treat (NNT; Laupacis et al. 1988) and the absolute numbers of people who benefited or were placed at risk. Any study that does not report these numbers should be viewed with great suspicion. The module ends with a discussion of systematic reviews and metaanalyses, and a final session is devoted to having the fellows search for and critique one from their field. Again, the rationale is that managers will often encounter these types of reviews and should know about their strengths and weaknesses. However, as the program has evolved over the years, we have found an increasing overlap with these methods in the first and second modules. Future iterations will most likely replace these sessions with new ones devoted to other aspects of design, such as biases that can jeopardize research findings. However, the final session – requiring fellows to critique a paper on their own – will likely remain an integral component.

Qualitative Methods The section of the module dealing with qualitative methods has a different set of preconceptions and issues to address. In contrast to the somewhat elevated expectations that managers may have of quantitative research and that need to be “demystified,” opinions of qualitative research can range from dismissal through indifference to a more guarded appreciation of what these methods have to offer. A few of the trainees come with an enthusiastic attitude toward the subject, but they have rarely articulated the value of this type of research for their managerial practice. Even fellows who are most familiar with the methods may have no clue about what those methods have to offer in the context of their EXTRA training. The goals for these sessions, then, are to explain a different way of doing research and to make credible the claim that qualitative research will be of great value to them in this program and in their future practice. For fellows, it is a matter of opening their eyes, rather than making them more skeptical. The findings from quantitative research are often used to make a decision about what programs or policies are likely to work. The findings from qualitative research are more useful for framing and understanding a problem, considering how to go about implementing change, and evaluating what that change means for those

Building Research Literacy

61

who are involved (Davidson et al. 2008; Goering et al. 2008). This means a fundamental shift from thinking about effectiveness to thinking about understanding and implementation. The content of this section includes a general introduction to qualitative research and to case studies, more in-depth experiential learning about two types of data collection methods (in-depth interviews and focus groups), and a session focused on critical appraisal of a journal article to convey distinctions about quality. There is some introduction in the previous module to the relevance of this subject matter to the fellows’ intervention projects. The fellows have been told that this knowledge will help them to define the nature of the problem and its meaning from various perspectives, as well as helping them to consider and select strategies for change. For clinicians working in health care delivery settings, this approach means recognizing that medical science, as traditionally defined, needs to be broadened to include theories and methods from the social sciences. It is less of a leap, perhaps, for those in community and government settings, where population health frameworks and participatory research methods are more common. The fellows come to this topic with a wide range of knowledge and experience regarding qualitative research, which makes it difficult to design and deliver an educational curriculum that is appropriate to the group’s learning needs. A basic introduction will be boring for those who are more familiar with the subject; on the other hand, moving quickly to more sophisticated distinctions between different types of grounded theory runs the risk of leaving behind those fellows who are still scratching their heads about how a study with a sample size of five can ever be taken seriously. As well, the level of understanding about qualitative methods has varied from year to year, with an apparent increase in prior exposure and the degree of comfort over time. Perhaps this reflects a growing acceptance within the wider community of qualitative evidence (Streiner, 2008). To help deal with the varying and unknown amount of the fellows’ past exposure to qualitative research, the introductory session includes an exploration of this very issue. The fellows are asked to place themselves on a continuum of experience from complete novice to expert. Although it is unusual for anyone to self-identify as an expert, those with more experience usually raise their hands as being “more than beginners.” Three or four of these individuals are then asked to

62

The Curriculum Elements

describe briefly one of the projects that they have been involved in, with the instructor asking questions to scope out the basic research questions and methods that were used. The instructor then appoints these individuals as “co-teachers” and turns to them for elaborations about their projects and responses to questions as the didactic content is covered throughout the session. Some examples in previous sessions have included a nurse who did her thesis interviewing patients in a prenatal clinic about their care and a regional health manager who participated as a team member in a case study of priority setting by advisory boards. By using real examples that are relevant to the learners, this approach illuminates what qualitative research is. It also takes advantage of the experience of the more advanced fellows, engaging them as resources for enhancing the education of their peers. The only disadvantage of this technique is that it takes more time than the more traditional structured-lecture format and interjects a degree of uncertainty and “messiness” into the teaching/learning process. Topics and questions that are raised by the examples may take the discussion down paths that differ from the straight and narrow outline in the prepared PowerPoint presentation. This necessitates a high degree of agility and skill on the part of the translators, which, thankfully, they have consistently demonstrated to date. Those who teach this module face the daunting task of covering in a few sessions a vast field of knowledge that is unfamiliar to some but not to all. Instructors are required to focus on a few selected topics that are most relevant to the particular group of trainees. In this segment the qualitative paradigm at a high level of generality is contrasted and compared with the quantitative paradigm. Emphasis is put on how the two approaches are compatible and complementary, rather than in opposition (Goering and Streiner 1996). The many different kinds of theoretical underpinnings and schools of thought are touched on only in a superficial manner – an approach that many academic researchers would likely find abhorrent. The module emphasizes that the methods are based on different philosophical assumptions about the nature of reality and of knowledge, but its focus is on the application of qualitative methods to managerial practice. Because case study designs are common in organizational change literature, the session includes a description of their attributes. The second and third sessions are devoted to in-depth interviewing and to focus groups. Fellows will often encounter these methods in

Building Research Literacy

63

the research literature, and they may find them to be useful datacollection strategies for their intervention projects as well. The best way to help fellows understand how these methods work is to give them direct experience. An in-class demonstration of a focus group is organized, with the instructor as the group leader and six to eight trainees as group participants. The rest of the class is asked to observe one of three aspects of the focus group: the content of what is said, the performance of the leader, or the non-verbal communication. After a brief overview of the methods, the focus group has a fifteen- to twenty-minute discussion, followed by the feedback of the observers. This has proven to be an effective teaching approach that elicits a lot of interest and involvement in how qualitative data are generated and underlines how facilitation and observation skills are critical to their quality. Finding a topic for discussion in the focus group that is appropriate and that yields sufficient, interesting interaction has been a challenge. Initially the entire session was set up in a playful manner, using the television show Survivor as a theme and positing that the three different professional groups that were selected for inclusion in the program were like tribes, with their own customs and desires to succeed in the game of EXTRA. This worked like a charm with the first cohort of fellows, who entered into the metaphor with zeal and had a good discussion about the ways in which their tribal identities might be influencing the group interaction. But the next two cohorts did not like the analogy, and many fellows objected to being forced into a discussion topic that was not relevant to their experience. This feedback helped the instructors to understand that the early stages of group development, when a cohort was just beginning to form, was not a good time to be emphasizing, even in a playful manner, the differences in identities and the possible tensions and rivalries that might result. A reframing of the theme and topic of discussion to one of inter-professional education, without prejudicing it as a divisive or competitive phenomenon, was a solution that was successful with cohorts four and five. The module’s final session elaborates on the different criteria that are used in qualitative research to judge quality and uses a small group exercise to apply a critical appraisal framework to a qualitative research article (Spencer et al. 2003). Again, this approach is a major shift from the appraisal of quantitative methods, which is rife with rules about how good research is conducted. There is a lot more

64

The Curriculum Elements

controversy in the qualitative field about how excellence is to be defined and recognized. Generally, there is more comfort with guidelines and frameworks for assessment than with checklists and prescriptions. In this session, a framework that was developed for assessing qualitative evaluations is applied to a research report, with each small group focusing on a subset of the criteria and questions. Groups then report back to the larger group with their impressions of the article and their experience with using the guide. This format, once again, encourages peer-to-peer learning, with the more experienced fellows assisting others in the group who are less comfortable with making judgments.

Health Economics The health economics session lasts an entire day (although some participants have said it feels like it goes on much longer!) The first two lectures introduce “abstract” economic concepts like scarcity, opportunity cost, and trade-offs, while the concluding two lectures provide fellows with opportunities to evaluate and use cost-effectiveness analysis from the scientific literature. The prejudice some fellows may have about applying economics to health care is addressed early on in the first lecture. Learners are asked to discuss a quote in the newspaper from a doctor-in-training saying, “I dread the day my decisions as a doctor will be based on economics.” Why would someone feel like this? Why would someone feel otherwise? After the ensuing discussion, we introduce the metaphor that economics is both an objective and a constraint. Objectives can include maximizing profit or patients treated or minimizing costs or inequity, while constraints place limits on needed resources like money, people, space, machines, and technology. Fellows are asked to consider the constraints they face while trying to do their jobs and suddenly economics does not seem so foreign. It is not difficult for them to identify how scarcity forces choice in their world. The main goal of the second lecture is to introduce the concepts of marginal benefit and marginal cost. We also try to explain why choices that equate marginal benefit to marginal cost are optimal. (For example, you should get another massage if the next massage visit is worth $100 to you and will cost you an extra $60. In contrast, if after your fifth massage that day, the next massage is worth $20 to you and will cost you an extra $60, it does not make sense to

Building Research Literacy

65

get that sixth massage.) Of course, the problem with most economic analyses in health care is that the extra benefit of a new treatment is not typically measured in dollars. For example, a new cancer drug will provide four more months of life for an extra $20,000. If students use a common benchmark that an extra year of life is worth $50,000, the new drug is not worth it (since it produces extra health at a rate of $60,000 for each extra year of life). However, if circumstances dictate that a more appropriate willingness to pay for an extra year of life is worth $70,000, then the new drug is worth it. The last two lectures are designed to encourage fellows to consider why a cost-effectiveness analysis does not usually indicate that something is cost-effective. The point we try to make is that whether the extra effect of the treatment is worth the extra cost depends on whether someone perceives the extra effect as valuable and to what extent. This, of course, is determined by one’s value system and not by the economic analysis. Alternatively, decision makers can link how much they are willing to pay to their budgets, buying the health care interventions and treatments that produce the most health per dollar until there is no more money. Thus, fellows learn that either the budget can determine their willingness to pay or their willingness to pay can determine the budget. Perhaps the greatest challenge faced with this topic is the potential letdown some fellows feel in realizing that the economic analysis does not make decisions for them. We emphasize the opinion of a famous health economist that “economic analysis should be used as a way of organizing thought rather than as a substitute for it.” The module stresses that the “right” answer is informed by health economics but should be based on the values manifested in the organization’s mission statement.

How Has It Worked? Feedback from the Fellows Planning a module while sitting in an ivory tower is easy; needless to say, it is vitally important that all of the fellows be informed – fully informed! – about everything in our area of expertise. However, since the premise behind EXTRA is that decisions should be informed by evidence, we as instructors and organizers of the program should really apply the same criteria to ourselves. To this end, we conducted an on-line survey of fellows who have finished the program and those still in the midst of it. We asked them what (if anything)

66

The Curriculum Elements

they remembered from each of the modules, but focused primarily on module 2. Were each of the components useful to them on the job? What would they cut from each section of the module? What would they add? Of the 127 potential respondents, we heard back from 44 people (35 percent); not a fantastic response rate by any means, but we likely heard from people who had strong feelings one way or another. An overall message gleaned from the feedback is that it is very hard to design a module for a group that includes people who come from a variety of backgrounds. For each of the sections, a number of fellows said, in essence, “Why are we being taught this? We studied it on the way to getting where we are.” Just as many people, though, said that the material was new and important material for them or that they had studied it previously but had promptly forgotten it. For example, for the section on quantitative methods, one person wrote that “I have never used [it] since the module. I would drop this entirely,” while another said, “This is crucial to the curriculum.” Similarly, while discussing qualitative methods, one respondent said, “It’s not likely that the fellows would be the ones leading these activities.” On the other hand, someone else wrote, “Because my work often involves qualitative methods, I was glad to see this section. It is often viewed as soft research, and I was glad to see those notions dispelled.” Another said, “I did not realize the importance of quantitative methods prior to this section.” When it came to the section on health economics, for every person who said something along the lines of, “I would consider dropping the entire area; it has not been relevant to my project or my role in my institution,” there was another who wrote, “I would have loved another day of this topic!” So, what have we learned? First, if you try to meet everyone’s needs, you’re doomed to fail. As we have said previously in this chapter (if not the book), teaching to a heterogeneous audience is a difficult, if not insurmountable, challenge. However, it is a necessary challenge to tackle if you want everyone to have a common core of knowledge. Second, if you want feedback, develop a skin that would be the envy of every elephant; teaching a group of managers is not a game for thin-skinned people! Although it is necessary to make use of the feedback to improve the sessions, you must nevertheless have a fair degree of confidence in what you do. Finally, adult learning requires a greater participation from the audience than simply

Building Research Literacy

67

listening to “talking heads.” Fellows enjoyed the time they interacted with one another and worked together on problems.

Looking Forward Where do we go from here? One obvious area is the need to develop a greater synthesis among the components within this module and across modules. At the current time, they are taught as separate sets of skills that must be mastered, at least to some degree. In the real world, though, they come into play simultaneously around any decisions that are made. In determining whether to open or (more likely in today’s climate) to close a unit, decision makers must weigh information from a variety of sources: quantitative research about the need for and the effectiveness of the program; economic studies about its cost-effectiveness or cost-utility; and the feelings and attitudes of those affected by and involved in the program. We try to simulate this scenario during the last session, when the fellows are broken into two teams. One team roleplays members of a hospital board presenting to residents of a small town (played by the other team) why they are closing down the obstetrical unit and relying on the services of a much larger, but distant, hospital. Although the presentations often call on research findings regarding morbidity rates in small centres, economic considerations, and the experiences of other hospitals, two things quickly become apparent: the data you need are never there, and the interpretation of data is never sufficient in its own right when feelings and emotions are concerned. The fellows obviously enjoy this session very much, but the integration of the various arguments – in qualitative research jargon, the “triangulation” of findings – is not explicitly formulated. We must find ways of showing that quantitative and qualitative research, along with health economics, are complementary ways of looking at problems; each may reveal a small part of the metaphorical elephant, but the composite picture is more complete than any one of the views in isolation. Examples do exist in the literature (for example, Elliott et al. 1997; Taylor et al. 1991) and these should be integrated into the module. Further, although this final session highlights the need for public deliberation and the use of consensus methods, it does so only by illustrating – quite dramatically at times – the tensions that arise within health care organizations, and between them

68

The Curriculum Elements

and the public when deliberations do not occur. Part of the training of any manager should explicitly include these elements. Finally, the other tension that exists within the module, and the program as a whole, is the desire to present as much information as possible, balanced against the fellows’ need for time outside the classroom to read, discuss, think, and (occasionally) sleep. There are components we would like to add – in particular, technology assessment – but to do so would require us to cut back on other things. Discussions about the future direction of the EXTRA training program must include what we can cut to make room for these additions. Most likely, these decisions will involve discussions not only amongst the faculty but also with graduates, to determine what is most useful and what can be scaled back. Overall, our impression, bolstered by the feedback from the fellows, is that this aspect of training is useful and needed. Continual fine tuning is necessary, which we hope will only increase the module’s usefulness for the fellows.

References Davidson, L., P. Ridgeway, S. Kidd, A. Topor, and M. Borg. 2008. Using qualitative research to inform mental health policy. Canadian Journal of Psychiatry 53:137–44. Elliott, S.J., S.M. Taylor, C. Hampson, J. Dunn, J. Eyles, S. Walter, and D.L. Streiner. 1997. “It’s not because you like it any better...”: Residents’ reappraisal of a landfill site. Journal of Environmental Psychology 17:229–41. Goering, P., K.M. Boydell, and A. Pignatiello. 2008. The relevance of qualitative research for clinical programs in psychiatry. Canadian Journal of Psychiatry 53:145–51. Goering, P., and D.L. Streiner. 1996. Reconcilable differences: The marriage of qualitative and quantitative methods. Canadian Journal of Psychiatry 41:491–7. Haynes, R.B., D.L. Sackett, G.H. Guyatt, and P. Tugwell. 2006. Clinical epidemiology: How to do clinical practice research. 3d ed. Philadelphia: Lippincott Williams & Wilkins. Laupacis, A., D.L. Sackett, and R.S. Roberts. 1988. An assessment of clinically useful measures of the consequences of treatment. New England Journal of Medicine 318:1728–33.

Building Research Literacy

69

Norman, G.R., and D.L. Streiner. 2007. Biostatistics: The bare essentials. 3d ed. Toronto: B.C. Decker. Shadish, W.R., T.D. Cook, and D.T. Campbell. 2002. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton-Mifflin. Spencer, L., J. Ritchie, J. Lewis, and L. Dillon. 2003. Quality in qualitative evaluation: A framework for assessing research evidence: A quality framework. Government Chief Social Researcher’s Office: National Centre for Social Research. Available at www.policyhub.gov.uk/ evalpolicy. Streiner, D.L. 1998. Thinking small: Research designs appropriate for clinical practice. Canadian Journal of Psychiatry 43:737–41. – 2005. With apologies to Albert: Everything is not relative [Editorial]. Evidence-Based Mental Health 8:93. – 2008. Qualitative research in psychiatry [Editorial]. Canadian Journal of Psychiatry 53:135–6. Taylor, S.M., S. Elliott, J. Eyles, J. Frank, M. Haight, D.L. Streiner, S. Walter, N. White, and D. Willms. 1991. Psychosocial impacts in populations exposed to solid waste facilities. Social Science and Medicine 33:441–7.

6 Leadership Skills for EvidenceInformed Decisions terrence sullivan , margo orchard , and muriah umoquit

Introduction The role of health care leaders is changing from managing cost pressures in health care transactions towards an imperative to reduce the “know-do” gap as a way of driving quality and performance. This change is fuelled by rapid changes in information technology and the explosion of information and knowledge. Consequently, the landscape is evolving rapidly with the downsizing of certain features and the upsizing or aggregating of other programs. In short, the health care environment is experiencing ongoing reorganization and is constantly changing. This environment is also paired with calls for greater accountability, sustainability, and the use of evidence in decision making. As Lamarche et al. and Bell suggest in the first two chapters in this volume, managerial decisions cannot be founded on trial-based evidence alone, and there are increasingly better sources of evidence upon which policy-makers and institutional decision makers can base judgements (Lavis et al. 2005). A health care organization as a whole, in contrast to individual clinical decision makers, will be more interested in active management and improvement of overall quality and system performance (aggregate levels of evidence). These types of evidence, which rely on complex data systems, are critical for measuring and monitoring health system performance and can improve quality, encourage change, and increase accountability (Hewitt and Simone 2000). Effective health care leaders must learn to align their leadership to make improvements in both aggregate and individual levels of evidence.

Skills for Evidence-Informed Decisions

71

Because health care is a complex form of work with multiple professional, labour, and stakeholder groups, there is no universal way of using evidence and evaluating performance. Adaptive leadership is needed, and the interests of a wide range of stakeholders need to be considered when evaluating the performance of a health care system. The methods used in the evaluation depend on the context and will vary depending on the stakeholders involved (Champagne et al. 2004). Because there are many stakeholders involved and because they cross between local, regional, and provincial contexts, it is essential for health care leaders to take on multiple roles and build organizational, clinical, and political bridges across these contexts (figure 6.1). It is no longer enough for individuals to understand and use evidence. Health care organizations need to build the culture and capacity for the use evidence within an organization. Doing so requires strong leadership (Dickson et al. 2007) and active processes to align leadership at different levels (Sullivan et al. 2008) to motivate performance improvement. Along with a culture of leadership driven by evidence, leaders must foster an organizational culture of learning, effective teams, and better use of information technology to improve health care quality (Ferlie and Shortell 2001). What follows are some of the leadership attributes reviewed in module 3 of the EXTRA program.

Leadership and the Use of Evidence The effective leader must work with a range of existing leaders, which requires a flexible approach and different styles of leadership suitable to different circumstances or tasks (Leatt and Porter 2003). For example, transactional and transformational leadership are two styles that can both be valuable, depending on the task at hand. Transactional leaders maintain organizations through employees or followers that act to maintain the status quo. The transactional leader’s goal is to maintain order, get jobs done, direct activities, reward performance, penalize non-performance, and ensure that subordinates have the skills needed for the task (Bass 1990). In contrast, transformational leaders aim for major change and see leadership as a means of social influence to increase the equality between leaders and followers. These leaders have an awareness of the beliefs, values, attitudes, and emotions that motivate transformation (Reinhardt 2004). While the transactional leader has been compared to the

72

The Curriculum Elements

Knowledge Centres

Board

Leader

Clinical Leadership

Healthcare Organization B

Healthcare Organization C

Executive Team

Broader Sociopolitical Environment

Research Organizations

Staff Host Healthcare Organization

Healthcare Sector /Region

Provincial Context

Figure 6.1 Adaptive leadership in the complex organization

crisis manager, “putting out fires,” the transformational leader is seen as the visionary who prevents the fires from starting in the first place or, alternatively, who creates a crisis in order to create a needed change (Trott and Windsor 1999). Closely aligned with transformation leadership is the idea of knowledge leadership. Knowledge leaders are analytical, innovative thinkers who utilize evidence-based techniques in goal-setting and performance measurement and who have financial skills and a strategic orientation (National Center for Healthcare Leadership 2005). These knowledge leaders strategically position the organization, manage change and stability, evaluate programs for quality and performance improvement, and get to know their enterprise by developing locally useful information (Eric and Vincent 2003). In addition to these leadership styles, a number of individualist intelligence types are relevant at different times for health care leaders. For example, analytical (Wechsler 1950), social/cultural (Cantor and Kihlstorm 1987), innovative (Sternberg and Lubart 1995, 1996), and intuitive (Anderson 2000; Keegan 1982)intelligence have all been discussed in relation to leadership. While a high conventional-intelligence quotient may buy entrance into the leadership fold, it is now believed

Skills for Evidence-Informed Decisions

73

that what separates the wheat from the chaff is the level of a person’s emotional quotient (EQ) (Goleman 2000). EQ involves the ability to use one’s own effective state to tap the effective state of others to accomplish objectives (Goleman 1998; Dulewicz and Higgs 2000). It is believed that both effective individuals and effective teams have high EQ and that this form of intelligence can be acquired (Urch and Wolff 2001). Although there is no single characterization of effective leadership style or intelligence, the most effective leaders have some awareness of their strengths, their developmental needs, and their continuously changing and complex context. Adaptive leadership, or the capacity to switch styles depending on the situation, is a relatively new characterization of the skills required for the modern organization (Heifetz and Linsky 2004). Complex leadership theory characterises the leaders’ adaptive force within an environment of turbulence and change. Key to this leadership model is fostering network relations through the middle to the top of the organization, encouraging collaborative and creative problem solving from the bottom up, and by nurturing systemic thinking, enabling interactions to emerge across the entire organizational system (Ford 2009). Given the unique context the health care environment, leadership requires the mastery of specific competencies to successfully advance change. These competencies, which are explored in module 3, provide the basis for improved self-knowledge.

The Competency Domains of Leadership: Transformation and Strategy, People, and Execution The National Center for Healthcare Leadership (NCHL) uses its Health Leadership Competency Model to help fulfill its mission of improving health system performance through effective health care management leadership. This model groups twenty-six competencies into three domains: transformation, execution, and people. Transformation involves organizational change and adapting these changes, execution requires enacting these changes, and people are fundamental in supporting both transformation and change (National Center for Healthcare Leadership 2005). We have identified six core competencies, which we have grouped into the three broad NCHL domains of transformation strategy, people, and execution (table 6.1). They include

74

1 2 3 4 5 6

The Curriculum Elements

Leadership and effective communication Seizing strategic moments Clinical and collaborative leadership Dyadic leadership: coaching and mentoring Leadership in groups: peer-assessment in leading people Leadership in managing organizational politics: alignment for decision making Transformation and Strategy

leadership and effective communication Effective communication, which is an important quality in leaders (Leatt and Porter 2003), is most pronounced in crisis situations. A health care leader must use effective oral and written communication techniques and group facilitation skills that can be catered to a wide range of constituents (Kreuter and Wray 2003). In health care, two very important communication skills are conveying a shared vision and being able to communicate crisis. Health care leaders must develop channels of communication with internal and external stakeholders, promote the sharing of knowledge through a free flow of information, and be capable of persuading and convincing others to support their recommendations or their point of view. A shared vision sets the stage for concentrated, collaborative (internal and external), and mutual effort. Communicating clearly defines goals, provides a framework for defined assignments and responsibilities, and provides a yardstick by which success can be measured (Turning Point 2001). It is just as important for health care leaders to be able to mobilize and motivate others during a time of crisis. They need to be able to create a “burning-platform” metaphor to capture the teams’ attention and focus on required action. The ability to communicate a threat, as well as the opportunity to overcome it, requires transformational leadership. This capacity involves letting participants know that change must happen and that it must happen soon; urgency can create the momentum necessary to begin and sustain change (Turning Point 2001). seizing strategic moments Closely allied with the capacity to communicate effectively is the capacity to initiate and exploit strategic moments. This requires a  planned approach to anticipate “where the puck will be,” to

Skills for Evidence-Informed Decisions

75

Table 6.1 The Six Competencies Transformation and Strategy

People

Execution

Leadership and effective communication

Clinical and collaborative leadership

Effective leadership in groups: peer-assessment in leading people

Seizing strategic moments

Dyadic leadership: coaching and mentoring

Leadership, managing organizational politics

borrow the often-cited Wayne Gretzky metaphor, and is essential to strategic initiative. In addition to anticipating and planning strategically opportune moments, it is also important to seize or create periodic windows of opportunity. Such moments are important in the world of policy but also in acute care and public health organizations. The seven leverage points of leadership used by IHI offer a useful guide to how to “lever” key points and moments of change in an organization (Reinersten et al. 2008). In the field of cancer we are now facing the dramatic and happy challenge of some long overdue improvements in survivorship. This has created the need to respond to the growing number of patients who require long-term episodic investigation, treatment, and follow-up. This is the kind of challenge that Ms. Laframboise identified in her EXTRA intervention project as the starting point for her region-wide efforts in dealing with the small fraction of chronically ill and fragile people in her region. A culture of evidence can help health care leaders anticipate and be ready to seize strategic moments like these. People clinical and collaborative leadership Leaders must generate organizational improvements by motivating and assigning people according to a central vision. Clinical leaders face one great difficulty in the transition from clinician to leader. Although in the past individuals were either clinicians or administrative leaders, when clinicians assumed the role of leader/ administrator, they often abdicated their clinical position to become an administrator only (usually without experience or the benefit of

76

The Curriculum Elements

formal training), a change in role that was seen as some kind of abdication or retirement (Schwartz and Pogge 2000). However, it has been suggested that remaining clinically active can help ensure success in moving to a leadership position. This way, clinical leaders are able to demonstrate the skills necessary for administrative and cultural leadership (Scheartz and Pogge, 2000), retain credibility, and use their skills in decisions that can enhance both the quality and cost-effectiveness of care. Change management is one of the most difficult tasks for clinical leaders, but it is an important part of clinical and collaborative leadership. Schwartz and Pogge (2000) identify three types of barriers to change that the clinical leader faces: systematic change (which requires people to step out of their comfort zones); behavioural change (which is based on management limitations or biases that lead to a tendency to lose objectivity or inflate the positive nature of the change); and political change (which is based on a reluctance to share information or arises from the distribution of skewed information). In the face of these interrelated barriers, clinical leaders must objectively choose the best path for the organization and champion that choice. They can do this through strategic and tactical planning, strong communication and negotiation, team-building, conflict resolution, and interviewing (National Center for Healthcare Leadership 2005). Clinical leaders must also make use of good research evidence, since its presence has been shown to enhance group consensus, a result that would ease the transition from clinician to leader (Lomas et al. 1988). There is evidence that collaborative leadership and the ability to work cooperatively with others improves health outcomes (Turning Point 2001). For example, Shortell and Schmittdiel (2004) and colleagues studied the performance of prepaid group practices in the United States. These practices, aside from working with a capitated model, are characterized by multi-specialty, collaborative teams of providers. Executive leadership from the highest-performing prepaid group practices reported that this collaborative model resulted in care delivery that was patient-centred, efficient, and effective and that made effective use of information technology. dyadic leadership: coaching and mentoring Along with cooperating with clinical colleagues and peers, another important skill for health care leaders is coaching and mentoring.

Skills for Evidence-Informed Decisions

77

Mentoring is akin to role-modelling, where the protégé sees attributes, qualities, or abilities in the mentor that he or she wishes to learn or emulate. Coaching is thought to be distinct from mentoring and involves a partnering of two equals: the coach and the person coached have a perceived parity of power. Coaching provides novices with the opportunity to develop their leadership skills, helping them to align their careers with the future of the organization (Donner and Wheeler 2004). Mentoring and coaching are two means of developing talent that serve to strengthen the human capacity of an organization, helping to create an energizing organizational environment. Effective health care leaders are developed, supported, and advanced by other leaders through mentoring and coaching. Execution leadership in groups: peer-assessment and leading people Each of the leadership competencies described above needs to be exercised in different contexts (table 6.1). Consequently, health care leaders need to be flexible, to be comfortable in a variety of roles, and to hold various teams and members accountable for outcomes (National Center for Healthcare Leadership 2005). Despite the individual aspects of leadership, it can and should be developed in teams (Leatt and Porter 2003). With increasing pressures and demands placed on the CEO, the model of the CEO as the alpha leader is changing, and CEOs are increasingly relying on their senior management teams for direction and shared leadership. Although this can be challenging, effective leaders are able to harness the ambitions of their senior leadership teams by aligning them to work together to meet the organization’s goals (Wageman et al. 2008). Acting as a catalyst, the leader is able to nurture and empower middle managers, accessing strengths and the assets of the team and building shared responsibility and accountability. To be most effective in leading groups, leaders must have some awareness of their personal leadership styles, strengths, and developmental needs. This can be informed by peer-assessment and the use of 360–degree assessments of leadership. Effective leaders can then address their shortcomings through self-directed learning and by attempting new leadership approaches when necessary.

78

The Curriculum Elements

leadership in managing organizational politics Health care leaders are faced with charged local, clinical, and regional political environments. Today’s leaders must use political skills to move an agenda forward, employing various tactics of influence as the situation requires and recognizing opportunities, barriers, and the necessary trade-offs that must be made. Furthermore, they must be strategists who position their organization for the future and understand the decision-making structures in their organization, and they must identify the real decision makers and their influences (National Center for Healhcare Leadership 2005). Strategic leaders are seen as investing in the future by nurturing people from all levels of the organization to be leaders. Political intelligence requires social awareness, the ability to communicate well, a high tolerance for uncertainty, and the ability to think clearly when under political pressure. Four underlying dimensions of political skill are networking ability, apparent sincerity, social astuteness, and interpersonal influence (Ferris et al. 2005). Strategic leadership can be seen in a hospital environment where the CEO must foster quality care by aligning both clinical professionals and the hospital board (Corbett and Mackay 2005). Strong governance is crucial, since the CEO manages the interests of both of these groups. While the board is ultimately responsible for quality and safety, individual physicians are also accountable for the quality of care they are providing in the hospital. Balancing these responsibilities and accountabilities requires strategy and political skill. In the course of module 3, fellows are exposed to multiple views of leadership styles and competencies. They focus on concrete examples provided by faculty spanning the six competencies. They also receive a profile of their own leadership style. They participate in group exercises and role playing to elaborate these competencies, including an executive decision simulation that allows them to provide feedback to each other on these leadership competencies. the changing world of leadership In their book Learning from High-Performing Systems: Quality by Design, Ross Baker and colleagues (2008) studied the attributes of high-performing health care systems and found a series of common attributes, one of which was leadership. Some of the elements of leadership were identified as strong administrative leadership, being a role model for organizational values, leadership that celebrates and participates in performance improvement activities, an

Skills for Evidence-Informed Decisions

79

emphasis on developing and fostering clinical leadership champions, board support and expectations on improvement initiatives and results, and continuity of expectations through the board, despite change in administrative leadership. Some of the best-performing hospitals in the United States and regions in Canada foster such leadership cultures. Their cultures are inquiring, aspiring, and collegial, and they do not try to defend their mistakes. Rick Shannon at Allegheny General in Pittsburgh was able to reduce the number of central-line infections in his hospital from forty-nine per year to none. By using leadership as a key lever to quality improvement and by closely studying defects and mistakes, he was able to foster a culture of quality improvement at the hospital (Kenny 2008). Similarly, Uma Kotagal’s work at the Cincinnati Children’s Hospital has been applauded for engraining quality improvement and excellence into its culture. By relying on evidencebased guidelines, patient-centred care, and the effective use of data, Cincinnati Children’s has become a leader in the quality improvement movement (Kenney 2008). Along with strong leadership, ensuring institutional alignment, stakeholder engagement, and buy-in from clinicians and clinical leaders are crucial elements of high-performing health care systems (Sullivan et al. 2008). In Canada, the former Calgary Health Region was an integrated health region that focused on innovation and quality improvement. Their transformation was facilitated through reliance on integrated information systems that linked process, outcome, and performance data. Part of the success was also a result of engagement of front-line staff and clinical leaders. Rather than focusing on a centralized “spider” model of leadership (in which the central leaders make all decisions), they adopted a “starfish” model, in which front-line staff and clinical leaders were all engaged and had shared ownership over the program’s success (Baker et al. 2008). Although a culture of quality improvement applies to entire organizations, there are also examples of individual clinicians who have applied a culture of quality improvement to medicine and started a movement to transform the health system. Atul Gawande (2007) identifies how individual physician compliance and adherence to protocols can raise the quality bar. Similarly, Charles Kenney (2008) highlights the contributions of some of the pioneers to the new quality movement. Leaders such as these, however, cannot work in isolation. Fostering effective teams and building organizational culture are especially

80

The Curriculum Elements

important for improving the quality of care in the health sector. Factors that improve a health care teams’ perceived effectiveness include a team focus on patient satisfaction, the presence of a designated champion, and the involvement of a physician in the group. Balancing the values of participation, achievement, openness to innovation, and adherence to rules and accountability are also important contributors.

Moving towards the Future As we look to the future, the EXTRA program can be improved to accommodate changes to the health system. Berwick describes two tracks of the new quality movement: an inside track, which is led by physicians and hospitals, and an outside track, which includes financing, incentives, and health system policy actors and structures (Kenney 2008). While the inside track has been well advanced, the outside track has not experienced the same evolution. Specific areas are currently under exploration, including funding based on appropriateness and quality, rather than funding based on volume and complexity. In order to effect change along this track, health care leaders will need to not only hold values of believing in evidence to guide decision making, but they will also need to understand the boundaries between institutional and policy leadership. By developing individual skills, and applying these skills in an evidencebased environment, leaders can begin to raise the quality bar. However, to really influence system-level change, the new health care leaders will need to understand the relationships between policy, funding, and institutional incentives. Improving outcomes, efficiency, and patient satisfaction remain central to the quality picture, but they operate within policy frames that are subject to intentional and political change. In this connection the expansion of EXTRA to take on a more active focus on the policy process and, potentially, a policy stream may be not only a desirable experiment but a central requirement to foster significant performance improvement over time in our delivery systems. The health care leaders of tomorrow must be able to navigate the team, the clinical setting, the board, and the broader regional, political, and policy landscape. Like our EXTRA fellows, they must lead, not as spectators, but as potent vectors for constructive and measurable change.

Skills for Evidence-Informed Decisions

81

References Anderson, J. 2000. Intuition in managers: Are intuitive managers more effective? Journal of Managerial Psychology 15(1): 46–63. Baker, G.R., A. MacIntosh-Murray, C. Porcellato, L. Dionne, K. Stelmacovich, and K. Born. 2008. High performing healthcare systems: Delivering quality by design. Toronto: Longwoods Publishing Corporation. Baker, G.R., P.G. Norton, V. Flintoft, R. Blais, A. Brown, et al. 2004. The Canadian adverse events study: The incidence of adverse events among hospital patients in Canada. Canadian Medical Association Journal 170(11): 1678–86. Bass, B.M. 1990. Bass & Stogdill’s handbook of leadership: Theory, research, and managerial applications. 3d ed. New York, NY: The Free Press. Cantor, N., and J. Kihlstrom. 1987. Social intelligence: The cognitive basis of personality. In Review of personality and social psychology, Vol. 6. Edited by P. Shaver. Beverly Hills, CA: Sage. Champagne, F., A.P. Constandriopoulos, and A. Tanon. 2004. A program evaluation perspective on process, practices, and decision-makers. In Using knowledge and evidence in health care, edited by L. LemieuxCharles and F. Champagne. Toronto: University of Toronto Press. Corbett, A., and J. Mackay. 2005. Guide to good governance. Toronto: Ontario Hospital Association. Dickson, G., D. Briscoe, S. Fenwick, Z. MacLeod, and L Romilly. 2007. The Pan-Canadian health leadership capability framework project: A collaborative research initiative to develop a leadership capability framework for healthcare in Canada. Ottawa: Canadian Health Services Research Foundation. Donner, J.G., and M.M. Wheeler. 2004. New strategies for developing leadership. Canadian Journal of Nursing Leadership 17(2): 27–32. Dulewicz, V. 2000. Emotional intelligence: The key to future successful corporate leadership. Journal of General Management 25: 1–13. Dulewicz, V., and M. Higgs. 2000. Emotional intelligence: A review and evaluation study. Journal of Managerial Psychology 15(4): 341–72. Eric, T., and M.S Vincent. 2003. Summary of the Review of Competencies and Best Practices in Healthcare Management. Presented at Meeting the Challenge: nchl’s First Invitational Symposium. Princeton, nj: 15 January. Ferlie, E.B., and S.M. Shortell. 2001. Improving the quality of health care in the United Kingdom and the United States: A framework for change. Milbank Quarterly 79(2): 281–315.

82

The Curriculum Elements

Ferris, G.R., S.L. Davidson, and P.L. Perrewe. 2005. Political skill at work: Impact on work effectiveness. Mountain View, CA: Davies-Black Publishing. Ford, R. 2009. Complex leadership competency in health care: Towards framing a theory of practice. Health Services Management Research 22(3): 101–14. Gawande, A. 2007. Better: A surgeon’s notes on performance. New York: Metropolitan Books. Goleman, D. 1998. What makes a leader? Harvard Business Review (November-December) 93–102. – 2000. Leadership that gets results. Harvard Business Review 78: 78–90. Goleman, D., R. Boyatzis, and A. McKee. 2002. The new leaders: Transforming the art of leadership into the science results. UK: Little Brown. Harvey, M., and M. Novicevic. 2001. Selecting expatriates for increasingly complex global assignments. Career Development International 6(2): 69–86. Heifetz, R.A., and M. Linsky. 2004. When Leadership Spells Danger. Educational Leadership 61(7): 33–77. Hewitt, M., and J.V. Simone, eds. 2000. Enhancing data systems to improve the quality of cancer care. Washington: National Academy Press. Institute of Medicine. 2000. To err is human. Washington: National Academic Press. Keegan, W., 1982. Keegan type indicator form b. London, UK: Warren Keegan Associates Press. Kenney, C. 2008. The best practice: How the new quality movement is transforming medicine. New York: Public Affairs Books. Kreuter, M.W., and R.J. Wray. 2003. Tailored and targeted health communication: Strategies for enhancing information relevance. American Journal of Health Behavior 27(S3): 227–32. Lavis, J., H. Davies, A. Oxman, J.-L. Denis, K. Golden-Biddle, and E. Ferlie. 2005. Towards systematic reviews that inform health care management and policy-making. Journal of Health Services and Research Policy. 10(S1): 35–48. Leatt, P., and J. Porter. 2003. Where are the healthcare leaders? The need for investment in leadership development. Healthcare Papers 4(1): 14–31. Lomas, J., G. Anderson, M. Enkin, et al. 1988. The role of evidence in the consensus process: Results from a Canadian consensus exercise. Journal of the American Medical Association 259: 3001–5.

Skills for Evidence-Informed Decisions

83

National Center for Healthcare Leadership. 2005. Health leadership competency model: Version 2.1. Reinersten, J.L., M. Bisognano, and M.D. Pugh. 2008. Seven leadership leverage points for organization-level improvement in health care. 2d ed. Cambridge, MA: Institute for Healthcare Improvement. Reinhardt, A.C. 2004. Discourse on the transformational leader meta narrative or finding the right person for the job. Advances in Nursing Science 27(1): 21–31. Schwartz, R., and C. Pogge. 2000. Physician leadership is essential to the survival of teaching hospitals. American Journal of Surgery 179(6): 462–8. Shortell, S.M., and J. Schmittdiel. 2004. Prepaid groups and organized delivery systems: Promise, performance, and potential. In Toward a 21st century health system: The contributions and promise of prepaid group practice, edited by A.C. Enthoven and L.A Tollen. San Francisco: Jossey-Bass. Sternberg, R., and T. Lubart. 1995. Defying the crowd: Cultivating creativity in a culture of conformity. New York, NY: Free Press. – 1996. Investing in creativity. American Psychologist 51(7): 677–88. Sullivan, T., M.J. Dobrow, E. Schneider, L. Newcomer, M. Richards, L. Wilkinson, G.P. Glossmann, and R. Walshe. 2008. Améliorer la responsabilité clinique et la performance en cancérologie. Pratiques et Organisation des Soins 39(3): 207–15. Trott, M., and K. Windsor. 1999. Leadership effectiveness: How do you measure up? Nursing Economics 17(3): 127–30. Turning Point (2001). Collaborative leadership and health: A review of the literature. Seattle, WA: Turning Point. Urch, D.V., and S.B. Wolff. 2001. Building the emotional intelligence of groups. Harvard Business Review (March) 80–90. Wageman, R., D.A. Nunes, J.A. Burruss, and J.R. Hackman. 2008. Senior leadership teams: What it takes to make them great. Boston: Harvard Business School Publishing Corporation. Wechsler, D. 1950. Cognitive, conative and non-intellective intelligence. American Psychologist 5(3): 78–83.

7 Managing and Sustaining Change ann langley, karen golden - biddle , jean - louis denis , and trish reay

Introduction Understanding change management – what it is, how it works, and its importance to organizations – is a crucial part of fulfilling the objectives of evidence-informed health care improvement. However, we also know that in order to truly understand the rational, political, and symbolic management skills needed to sustain organizational change, we cannot rely on evidence alone. Several chapters in this book emphasize the importance of mobilizing research evidence to improve health care systems and organizations, and all provide tools for helping managers and policy-makers to achieve this end. Our chapter aims to place the use of evidence in the context of managing change: examining how change occurs in organizations, how it can be effectively managed, and, once achieved, how it can be sustained. These topics are the focus of modules 4 and 5 of the EXTRA program. We hope in this chapter to provide insight into some of the ideas at the core of these modules, linking them to the challenges faced by fellows as they attempted to use their intervention projects to put evidence-informed management into practice in their organizations. We use case examples and a variety of exercises and simulation to advance the content of these two modules. Organizational change encompasses many dimensions, including structure, culture, technology, practices, people, and mental models. Many variables surround change, such as context, content, and process (Armenakis and Bedeian 1999), and there are specific change styles that may be adopted, such as coercive or collaborative styles (Dunphy and Stace 1988; Huy 2001). However, particularly relevant

Managing and Sustaining Change

85

to this chapter is the distinction proposed by Weick and Quinn (1999) that concerns the pace and rhythm of change. Specifically, these authors speak of “episodic” or “continuous” types of change.

Episodic Change “Episodic change” refers to radical, clearly identified and deliberate change initiatives that aim to transform the functioning of an organization over a condensed period of time (Weick and Quinn 1999). Underlying this concept is the idea that organizational structures, cultures, and practices tend to form “coherent configurations” in which these diverse elements mutually reinforce each other and operate in symbiosis with one another. Normally, organizations are seen as quite stable. However, over time they may tend to drift out of alignment with the environment, creating the need for a radical readjustment. Major “planned change” initiatives generally aim to ensure that a specific reform strategy is implemented, and they usually try to improve organizational functioning in some important way. Technological changes, restructuring, and mergers are examples of activities that tend to be viewed as having this episodic – as in responding to a particular episode or situation – character. There are many examples of episodic change in health care. The restructuring in the Quebec health care system in 1995 (Denis et al. 2006) provides a helpful example. Like many other provinces at the time, Quebec in 1995 was faced with rising costs and diminishing budgets. While there was a surplus of acute-care beds, there had been insufficient planning for strategies to deal with an aging population, such as investing in home care and long-term care or taking advantage of new technologies that enabled care and treatment in the home. To address this problem, the Quebec government asked the Montreal Regional Board to develop a plan that would reconfigure the regional health care system and save $190 million annually – a radical transformation that posed enormous challenges. Story 1: An example of Episodic Change: Health Care Restructuring in Quebec in 1995 On 15 May 1995, Jean-Robert Sansfaçon, the well-known editorialist of Le Devoir newspaper, wrote the following lines in an editorial reacting to a proposed major overhaul of the system of health care institutions on the Island of Montreal:

86

The Curriculum Elements

Faced with the prospect of slashing more spending each year, the Régie régionale de Montréal-Centre, flexing the expanded powers delegated by the National Assembly under the previous government, wants to propose reforms and is trying to build the consensus needed to take action. It is approaching this task with such surprising energy, openness and determination that it begs the question: Who are these people, this young president Marcel Villeneuve, to name just one, with the nerve to suggest shutting down nine hospitals and then sell the idea as an improvement in care? Have they lost their minds? Do they have the political authority and support indispensable to carry out their mission? We dare say they do because, although it may upset a lot of people, the reform is definitely needed. However, it would take a very naive person indeed to deny that in this referendum year, the Parti québécois government is sorely tempted to let a few volunteers rise to the cause while keeping open the option of backpedalling if these bold activists make the smallest misstep. What is the Premier waiting for to voice support for the initiatives of the Montreal Régie? This quote reveals the enormity of the challenge facing managers of the Régie régionale as they strived to implement the proposed overhaul. At the same time, since the quote was penned by an editorialist with a leading newspaper not known for its bland opinions, it reflects a measure of success in terms of public relations! Despite the disruption and social costs the reform was likely to incur, the print media appeared ready to lend its support and portrayed the main protagonists in a remarkably favourable light. And indeed, the restructuring ultimately did go through. Seven hospitals were closed on the Island of Montreal, and a host of other projects were launched. The overhaul included the transfer of resources to CLSCs for home care, amalgamation of institutions, and the introduction of a singleservice point for elderly people – all components of what was called the “shift to ambulatory care.” It is true that this shift was not without its pitfalls, owing in part to events that followed the actual restructuring plan. For example, the government negotiated very generous early retirement terms with the labour unions and medical federations as part of its zerodeficit objectives. As a result, the staff surplus caused by the hospital closings quickly turned into a shortage, with adverse consequences

Managing and Sustaining Change

87

that affected the entire system. Yet, at the time, the Régie régionale’s ability to actually obtain consensus for such radical change and even to acquire support for it from the media was truly remarkable. How did they achieve this? How did the regional agency use evidence in its decision making? What other processes were significant?

Continuous Change In contrast to the episodic change just described, the term “continuous change” evokes the evolving and emergent nature of life in organizations (Weick and Quinn 1999). Rather than seeing organizations as normally stable and punctuated by occasional shifts, the “continuous change” perspective draws attention to the everyday decisions, actions, interactions, and adaptations that are constantly occurring and modifying, sometimes imperceptibly, what the organization does and how it does it from one moment to the next. Over time, these small “incremental adaptations” can result in significant transformations, partly intended and partly unexpected, as people improvise in their everyday practices. Here, change is not dramatic but subtle and progressive. Yet it can also be deliberate, as people have the potential to stimulate, over time, many different incremental steps that will ultimately get them where they want to be. Our second story describes an example of this type of change in the health care arena: the development of a nurse practitioner role in Alberta between 1995 and 2005. Here we see how nurse practitioners managed to create a space for themselves alongside other professionals through their everyday activities, progressively moving forward through a series of “small wins” (the data in the following story are from Reay et al. 2006). Story 2: An Example of Continuous Change: Developing the Nurse-Practitioner Role in Alberta Nurse practitioners (NPs) are registered nurses (RNs) who have additional education, training, and experience and who are authorized to diagnose and treat health conditions and prescribe medications. Although nurse practitioners are common in the United States, they are less so in Canada. New Alberta legislation in 1995 allowed special nurses to provide “extended health services” in areas that were medically underserved.

88

The Curriculum Elements

This legislation was followed by a number of experimental projects in which special nurses took on increased responsibilities. The number of special cases grew over time, but it was not until 2002 that legislation was passed allowing the title of “nurse practitioner.” In January 2004, 92 NPs were formally registered with their professional nursing association. In 2005, there were 130. Their role is defined by a legislated framework that includes educational and work experience standards. NPs have clearly gained widespread acceptance from health care workers and the public as an important and effective new category of health care professional (Alberta Association of Registered Nurses 2000). Currently, most NPs work with physicians and other health professionals as part of interdisciplinary health care teams in critical care units, active treatment hospital wards, long-term care facilities, and primary health clinics. Achieving this change was not easy, and the process was driven by the nurses themselves rather than top-down. As one nurse practitioner describes the process: I guess when you’re trying to make a change in something that’s been established for such a long time, you just have to keep at it. It’s persistence, persistence, and persistence. And I think I probably need a little more patience, because to keep at it – that is tough – but then all of a sudden resistance backs off. I think we’re slowly getting there but I feel like I’ve been swimming uphill. We can see that this was a continuous change over several years that required persistent effort. But what exactly was done to achieve this positive result? What part did evidence play in this process? What other processes were involved in successfully developing the nurse practitioner role in Alberta? The type of change that these nurse practitioners accomplished is important for health care managers and professionals to understand and to be able to emulate. The example reminds us that achieving change does not necessarily require a big flashy program explicitly labelled “change” but, rather, persistent and directed energy and drive over long periods. For example, Sonja Glass’s intervention project (the introduction of a patient safety culture) could have been viewed as an episodic “big bang” one-time change. Indeed, as she notes in her prologue, many of the EXTRA fellows had initially created projects that were

Managing and Sustaining Change

89

rather daunting, if not overwhelming. Thinking about these projects as contributing to a long-term vision that could progressively move forward through continuous change was an enabling insight for many of the fellows, who began to look for ways to incorporate “small wins” by breaking their own projects down into more manageable parts. This action also enabled the projects to spread. Indeed, Sonja’s achievements in the area of patient safety are now being taken forward to another stage by Dr Todd Webster, a Grey Bruce EXTRA fellow in the program’s fifth cohort.

Three Processes for Successful Change: Rational, Political, and Symbolic Both of the stories of change are grounded in evidence of one form or another. In the Quebec case, evidence was available that the health care system was imbalanced and that increased investment in community care was needed. For the Alberta case, studies had shown that nurse practitioners could make a distinctive and valuable contribution to the health care system. Clearly, the underlying rationale for change was there, and yet in neither case was the presentation and dissemination of evidence alone enough to enable change. Indeed, if evidence had been the only consideration, nurse practitioners would have been adopted much earlier. Rational arguments based on evidence must be embedded in other kinds of processes, as noted by Lamarche and colleagues in chapter 1 of this volume. The basic underlying difficulty is that although evidence and analytical information can resolve factual uncertainties, they do not provide support for dealing with the ambiguities associated with implementing change in context. In particular, change initiatives are influenced by different and often divergent values and interests that permeate health care organizations. Moreover, they do not take into account how decision-making power is distributed among different stakeholders. Thus, it is important to be clear about what is meant by interests and values and how they influence the potential for change. Proposed changes generally involve risks and benefits that affect stakeholders differently. The term “interests” refers to the nature of these risks and benefits at the personal level. For example, it is clearly difficult for employees to see the closure of their hospital in a favourable light, whatever “rational” arguments may be put forward from an overall systems’ perspective. In contrast, those who work in

90

The Curriculum Elements

a CLSC in Quebec will generally see increased investment in community care as a positive development, because it is likely to benefit them. In any discussions between stakeholders about what should be done, their interests and the power relationships between them are critical to determining the outcome. Evidence may be helpful to a degree in supplying grist for the discussion; however, the evidence will be interpreted differently by different people, and it can easily turn into a political tool for people to use to promote their own position and agenda. The result is that no evidence is likely to be perceived as entirely neutral or trustworthy, and no would-be change agent can ignore the interests of different stakeholders or the power relationships between them. Change is partly a political process. Any use of evidence must be accompanied by a clear reading of the political landscape. All of this might lead one to believe that in making choices, the only thing that counts is “what’s in it for me?” While the consideration of personal interests is important, it is also clear that change is more successfully promoted (or resisted) if it can be grounded in deeply held “values” that justify the chosen position, rallying others to the cause for “good” reasons, as well as interest-based ones. For example, in the health care arena, most people share an understanding that the ultimate purpose of the health system is to benefit patients and the general public. Change that cannot be presented in some way as achieving these goals is doomed. Moreover, to make their positions acceptable to others, stakeholders will most often defend their positions in terms of values related to the common good, rather than in terms of personal interests. In general, this means that change agents need to take into account the “symbolic” dimension of change – what meanings are given to it and what values are associated with it. Change is also a symbolic process. Evidence alone is not sufficient to achieve change, but political and symbolic dimensions (associated with interests and values respectively) also need to be considered. This notion builds on what Allaire and Firsirotu (1985), as well as other change management scholars and practitioners, have observed in domains far removed from health care.

Rational, Political and Symbolic Management in the Quebec Case We can see that the Montreal Regional Board had its work cut out for it when it embarked on a process to implement a restructuring

Managing and Sustaining Change

91

that involved huge budget cuts and hospital closures. The very idea of such a change would seem to go against powerful interests in the health care system. As illustrated in Jean-Robert Sansfaçon’s editorial quoted above, the reform had serious value implications. As he put it, how could the regional board have “the nerve to suggest shutting down nine hospitals and then sell the idea as an improvement in care? Have they lost their minds?” This case clearly needed skillful political and symbolic management, as well as clear thinking and rational analysis. As demonstrated by the processes implemented by the Regional Board, rational, political, and symbolic management modes are intimately intertwined in practice. The form of the project initially put forward by the board illustrates the political dimension of change management. It would have been possible for the board to propose a variety of different scenarios for recouping the $190 million that it was required to cut from the regional budget. However, some scenarios were clearly more likely to generate political support than others. A particularly useful tool for assessing the political dimension of change projects is “stakeholder mapping,” as proposed by Scholes (2001). As shown in figure 7.1, stakeholder mapping involves placing key stakeholders on a two-by-two grid showing (1) their power to influence adoption of the change proposal and (2) their interest in doing so. Their likely position with regard to the project being considered “favourable” or “unfavourable” is represented by a plus or minus sign. In order to accurately place stakeholders on this grid, it is important to consider their various sources of power and interests. For example, elements that might be used to assess the power dimension for a stakeholder map might include formal authority, control of resources (expertise, budgets, prestige), and connections to powerful others (politicians, media). To assess interests, one should examine the consequences of the focal project for that stakeholder and also evaluate their previous public positions on an issue. The most useful way to use this grid is to compare alternative projects in terms of their political implications. The map shown in figure 7.1, though approximate and not exhaustive, suggests a number of useful things. First, it suggests that a solution where budgets of all health care organizations are cut equally across the board is not likely to find many allies, since the consequences would be negative for almost everyone. In contrast, a project in which small hospitals were closed and resources transferred to

92

The Curriculum Elements

Project 1: Cut all health care organizations equally across the board low

Interests

Power

high

Interests

high

CLSCs (---) Nursing homes (---)

low

Project 2: Close six small hospitals and transfer resources to CLSCs and nursing homes

Municipalities (-) General public (--)

low low

Power

Hospitals (--) Media (--) Unions (---) Doctors (---)

high

high

CSLCs (---) Nursing homes (+++) General public (--) Small hospitals (---) Municipalities (---) Large hospitals (+) Media (?) Unions (---) Doctors (?)

Figure 7.1 Application of stakeholder mapping to the Quebec case

CLSCs and nursing homes while leaving larger hospitals untouched would present a very different picture. Although CLSCs and nursing homes are not the most influential organizations in the health care system, together they would clearly represent an important constituency supportive of reform. We can see that the larger hospitals are more powerful, and they would also gain – although the benefits would be more marginal. At this point, it would be important to maintain their interest and mobilize them in supporting the reform. The grid also shows that the media and the medical community are key stakeholders. However, it is not clear at first which way they would lean in the second scenario. It would be important to accord them an appropriate degree of attention to ensure they understood and sympathized with the project. Clearly, a project that could be presented as respecting societal values would have more chance of attracting support from the media. With its emphasis on transforming the health care system, the second scenario appears more attractive than the first, at least from this perspective. In addition, the grid suggests that there are some stakeholders who might become more “interested” with the second scenario (more likely to try to influence the decision). For example, municipalities might not react much to the first scenario, but a different reaction could be expected in the second scenario from those municipalities that were associated with a hospital likely to be closed. This type of information is important to consider when developing proposals. Finally, some stakeholders (for example, the unions) would be likely to oppose both

Managing and Sustaining Change

93

solutions; this suggests that it might be important to address and mitigate their concerns through careful attention to the consequences of the proposals for employees. In summary, stakeholder mapping is useful for identifying which proposals are likely to acquire support, as well as for getting ideas about which stakeholders will need what types of attention as the change process evolves. As can be seen, it is in a certain sense a rational tool that builds on evidence about interests, values, and power relations to systematically consider political issues affecting change projects. In practice, the Montreal Regional Board did conduct its own stakeholder analysis and developed a strategy for orchestrating the change process on the basis of it. The project was submitted to public consultation in a series of two hearings. For the first set of hearings, the Regional Board presented a document that established the basis for the proposal and asked people to consider two alternative scenarios: (1) cutting across the board versus closing hospitals, or (2) transferring resources. The public-hearings process was a courageous strategy, but it was also both politically savvy and symbolically important. “Public-ness” ensures that stakeholders consider the public good, as well as private interests, in making their arguments. At the same time, the public nature of the process reaffirmed shared values of transparency and procedural justice in decision making. The Regional Board came out of the hearings with a fairly broad consensus on the idea that the second scenario for transformation was preferable. At this stage, the names of the hospitals that might be closed had not been made public, facilitating an early consensus on principles and avoiding an emotional debate. For the second set of hearings, the Regional Board named the hospitals that were targeted for closure. It was here that extensive use was made of an analytical or rational approach that was nevertheless still embedded in a political and symbolic context. The board developed a set of five indicators, based on data provided by the hospitals themselves over the previous three years, to calculate a performance score for each hospital and used this information to compose the list of targeted hospitals. There was a lot of discussion of these criteria during the public hearings, and the targeted hospitals presented a variety of alternative suggestions. The board rejected other suggested measures of performance because they were based on subjective judgments or they did not enable the decision to be

94

The Curriculum Elements

made: “In terms of criteria, those that we used, the five of them, are the most objective criteria that exist.” The use of these numbers in making this choice also had other effects at the political and symbolic level. First, the hospitals targeted for closure were hampered in their ability to argue against the proposals because a change in the decision meant a redirection of fire to a sister institution. This was made more salient by the fact that many of the hospitals had given their explicit support to the board’s broad proposal at the previous stage. Second, the system of indicators provided a means for organizations not targeted in the proposals to consent to the choice yet distance themselves from the decision, because it was determined objectively by the numbers. Beyond the technicalities of the numbers, the way in which members of the regional board behaved and presented themselves during the entire set of hearings clearly impressed many observers (including the editorialist quoted earlier). They appeared to be in search of the best solution to the problem at hand, they appeared willing to listen to suggestions and included them in their proposal if they could, and they showed competence in their knowledge of the health care system and of their own project. Thus, the change management process used clearly embedded rational, political, and symbolic processes in close synchronization. In terms of rational processes, the Regional Board adopted an analytical process in order to establish the list of hospitals to be closed and built on knowledge of the basic imbalance in the system to put forward the entire proposal. In terms of political processes, both the project itself and the way in which it was presented, in two separate stages, aligned the interests of key actors in ways that ensured support of a dominant coalition and that isolated the targeted institutions later in the process. Finally, in terms of symbolic management, the overall transparency of the public hearings, the apolitical stance of the Regional Board, and the mobilization of values such as quality of care and service to the community contributed to making stakeholders’ support seem not only expedient but also “right and good.” Major episodic change, evidence-informed or not, requires skill in political and symbolic management. A final point worth making here concerns the role of crisis (or as one facilitator in the module called it “the burning platform”) in allowing the initiation of radical episodic change. The project to move resources from acute care to community care in Quebec, as elsewhere

Managing and Sustaining Change

95

in Canada at that time, was long overdue. However, it was only when the province’s financial situation was recognized as reaching crisis proportions that the will and political support could be mobilized to do something about it. While problematic in itself, crisis can paradoxically be an ally in moving constructive evidence-informed change forward. Thus, a first task in initiating such change may involve persuading participants and observers that the platform is indeed burning and that a drastic intervention is required to save it.

Rational, Political, and Symbolic Management in the Alberta Case While the Quebec case illustrates rational, political, and symbolic management on a grand scale, the Alberta case study of the implementation of nurse practitioners illustrates how these processes are intertwined and mobilized in a more subtle fashion in everyday micro-processes or practices. An example of political processes in the Alberta case concerns how the nurse practitioners attempted to take advantage of opportunities that arose to legitimize their role. As Reay et al. (2006) noted, they acted as “political entrepreneurs” (Buchanan and Badham 1999) who tapped into their understanding of political dynamics in various settings and into prior social networks and current interactions with colleagues and the community, to cultivate opportunities to increase awareness and acceptance. For example, a shortage of medical residents was initially seen as an opportunity that would benefit the profession, since the nurse practitioner was clearly in demand. However, more was needed to ensure that the nurse practitioner’s role would not be simply as a “physician-extender” that would become redundant when the shortage disappeared. Thus, nurse practitioners lobbied to obtain legislation that would define their role as separate from physicians and more aligned with nursing, believing that this would place them in a stronger position to continue growing in numbers over time and to avoid control from physician groups. Clearly, the positioning of the role of nurse practitioner is a highly delicate political matter, since a number of different professional groups and subgroups partly share the terrain of expertise claimed by them. Reay et al. (2006) also report on how nurse practitioners acted to prove the value of their role with colleagues from different professions. However, this value was not demonstrated mainly by

96

The Curriculum Elements

presenting formal evidence on the benefits but rather by building trust in daily interactions, as the following quote from a nurse practitioner illustrates: “You have to build trust and be there and sometimes you have to be there more than you normally would be there just to be able to build that, you know, that you are there as a resource, that they can trust that you’re going to get things done.” Another manager describes how she worked with doctors to gain acceptance for the role, essentially by demonstrating that it could deliver results in terms of patient outcomes: “We just had to strategically come at it from [the perspective] that if we deliver a good product, we’ll be above the criticism.” In these interactions, the nurses mobilized their deep knowledge of what their colleagues needed and would appreciate to establish the legitimacy of the nurse practitioner’s particular expertise. Finally, in another example, nurse practitioners showed political savvy in the way that they moved to develop a model for “advanced nursing practice” at the regional level. They carefully selected people to be on the committee that designed the model to ensure that key stakeholders were represented, while deliberately excluding physicians to establish that this role lay within nursing rather than medicine. Knowing that a lack of support from physicians could derail the project, they nevertheless consulted them subsequently on the model, which was later approved at the regional management level. Similarly, in her project to implement a patient safety culture at Grey Bruce Health Services, Sonja Glass had to consider whether, when, and how to involve physicians in her initiative. The decision was delicate, given that the inclusion of physician stakeholders would make the project more complete but at the same time might raise barriers that could derail the promising momentum developing among nursing professionals. The involvement of a physician leader (EXTRA fellow Dr Todd Webster) in the second phase of the project proved helpful in legitimizing the change with physicians. Returning to the nurse practitioner case, the creation of the advanced nursing practice model itself can be seen as a form of symbolic management in the ongoing continuous-change process. The model established the existence of the role, defining and formalizing its content. In doing so, the model became a symbol representing the value of the advanced practice nurse as a distinctive nursing position – neither traditionally medical nor nursing – that expanded the traditional boundaries of nursing work through appropriation of some

Managing and Sustaining Change

97

medical work (Golden-Biddle and Reay 2009). Developing and incorporating the nurse practitioner job description within a region’s human resources systems formalized what a nurse practitioner was and meant, serving as a base of legitimacy for future actions. Overall, the nurse practitioner case illustrates how nurse practitioners individually and collectively moved in various ways to make their role count within the health care system, building successive moves on previous ones. Each move drew on their knowledge, experience, and contacts within the system and their abilities to navigate the political and cultural processes to achieve progress. Significant developments that provided a base for future actions, such as the advanced nursing practice model, were celebrated as achievements on a longer road to full recognition. Clearly, the process described here is more diffuse than that presented for the case of episodic change. However, in its own way, it does illustrate the interaction between rational, political, and symbolic processes to advance change. Nurse practitioner roles have been shown to be effective, and the cumulative evidence of their effectiveness no doubt partly lies behind the emergence of this role in Alberta and elsewhere. Yet the development and full legitimization of the nurse practitioner role depends to only a limited extent on that evidence. As illustrated in the direct quote used earlier, nurse practitioners had to work very hard. They had to become entrepreneurs in pursuing the development of their role and in positioning it with respect to other actors in the health care system in order for it to achieve taken-for-granted status.

Sustaining Change How can change be successfully sustained beyond the initial burst of energy? “Sustainability” is defined as “the stabilization over time of a change initiative through its incorporation into routine work processes and organizational norms coupled with ongoing investments in the development of capabilities to achieve higher performance, adaptation and innovation” (adapted from Shediac-Rizkallah and Bone 1998; Lawrence, Winn, and Jennings 2001). This definition begins to dissolve the boundaries between notions of episodic and continuous change. Indeed, it could be argued that any change should be considered to be “situated,” that is, embedded in a context of continuous evolution. A large structural transformation

98

The Curriculum Elements

such as that described in the Quebec case does not end when the decision is taken. Rather, the project itself necessarily interacts with its context over time as it is implemented. The ideas that formed the basis for change are transformed as this happens, sometimes reinforcing change intentions and sometimes diverting or diluting them. The Quebec case presented earlier illustrates some of the complexities of sustaining change. So far, we have only described how the decision to transform the health care system was achieved, placing particular emphasis on the dimension related to hospital closures. The hospitals in question were indeed closed. However, the most complex part of the overall reform project was not the hospital closures but all the other measures that had been proposed, in part to help legitimize those closures, as well as to invest further in community care. These measures included transferring human resources from the closed institutions to other organizations, developing improved integration among hospitals, CLSCs, and nursing homes, creating a single entry point program for the frail elderly, and providing twenty-four-hour-a-day availability of doctors in all parts of the region. All measures were intended to be implemented over several years following the decision. These complementary but important changes were relatively easily “approved” in formal terms by all stakeholders, but they were much harder to implement effectively because the process required large numbers of people to alter their ways of working. They also required continuous adjustments and reinforcements through time in order to become institutionalized in day-to-day operations. This was a more complex challenge, since some of the conditions of sustainability were missing. Similarly, in the Alberta case the nurse practitioners clearly succeeded in their initial efforts at political entrepreneurship, first, by positioning themselves as offering advantages to others (by reducing physicians’ hours of work and offering rapid response to nurses’ queries) and, second, by persistently and consistently explaining to others what their role entailed and how it could contribute. Yet after five years, many nurse practitioners still seemed to be engaging in these very same efforts. Some NPs complained that they still had to explain what a nurse practitioner is; there were always new people who needed persuading. Others were beginning to realize the downside of a strategy in which they positioned themselves, essentially, as providing advantages to other professionals. As one respondent noted:

Managing and Sustaining Change

99

I think we’re helping out physicians and it sometimes worries me a little bit that we do a lot of their discharge work. So, we’ll do the discharge summary – which is tedious, but it frees them up so that they can go see people waiting in Emergency, or review consults with the residents or fellows. That seems like a really good use of their time, but if it’s just so that they can go have coffee with their cronies while the NP is doing five or six discharge summaries, I don’t think that’s a good use of time. In other words, the change had not always come to be taken for granted, or if it had, it was not necessarily in desirable ways. Again, some of the factors of sustainability were probably missing. So what are these factors? Some are summarized below. The first category groups together the more formal, rational elements of change that often receive the most emphasis in formal plans. The second category examines the more human elements, which are associated with the political and symbolic dimensions of implementing and sustaining change.

Rational Management: Formalization, Accountability, and Monitoring In order for changes to become embedded in everyday practice, some kind of formalization in texts, structures, rules, and procedures is often necessary. Formalization creates a degree of irreversibility and at the same time entrenches desired changes in procedures that can be called on as a reference and that enable acceptance into the organization’s routine. For example, the moves by the nurse practitioners described earlier to formalize the model for advanced nursing practice at the regional level and to embed it in the human resource department’s policies was a necessary step in legitimating the role and contributed to its potential sustainability. Similarly, in their study of the implementation of organizational reforms associated with the hospital closures, Rodriguez et al. (2007) showed that the measures embedded in formalized protocols were also the ones that had the most impact in terms of modifying concrete practices. Those for which no formally agreed protocols were established essentially faded from view after sporadic and localized attempts to make them work. Similarly, evidence-informed change is more likely to stick if relevant indicators have been put in place to explicitly monitor

100

The Curriculum Elements

improvements and their impacts. These indicators may also play a reinforcement role by demonstrating to key stakeholders the benefits achieved by implementing change. Unfortunately, sometimes a mismatch occurs between the ostensible objectives of the reforms and the indicators that actually count in performance evaluation and decision making. Asking for B while measuring and rewarding A is a classic error that can undermine otherwise laudable attempts at improvement (Kerr 1975). While such rational elements are important, they are not in themselves sufficient. Putting in place formal rules and procedures, establishing new structures, and monitoring results may create a new playing field. However, the way in which the game will develop will depend as much, and probably more, on the people involved and how they are motivated, empowered, and involved in the change and on what this means for them in terms of their values and interests.

Political and Symbolic Management: Empowering and Mobilizing Networks of Change Champions Sustaining change initiatives requires continuous investment in material and human resources. As mentioned earlier, change initiatives are situated; they are always renegotiated in action. One simply cannot expect that once a formal mandate to implement change has been launched, everything will be implemented as planned. For one thing, changes in behaviour are unlikely if violating personal interests is involved, especially where powerful professionals and organizational opinion leaders are concerned. For example, in the Quebec case it turned out to be very difficult to associate physicians with some of the otherwise laudable measures aimed at improving access to care following hospital closures, since the measures seemed to constrain the physicians’ well-established autonomy (Rodriguez et al. 2007). This is not an uncommon story. It is interesting to note that the United Kingdom’s National Health System has been more successful in mobilizing physician champions through certain forms of incentives, such as the pay-for-performance system in primary care (Doran et al. 2008). There is a clear need to reflect more on how changes affect the various people involved and to consider how incentives might be creatively manipulated to enable those who have the most to offer to join in or even help to lead and champion the effort. It has

Managing and Sustaining Change

101

been repeatedly shown that clinical leadership can make a significant difference between successful and unsuccessful change (Berwick 2003; Ham 2003; Vaughn et al. 2006). Beyond the issue of interests, Balogun and Johnson (2005) showed how middle managers in a restructuring attempt were forced to engage in “sense-making” to understand the change, work out what it meant for them, and find ways to cope with it in concrete actions over a long period. Indeed, any change may generate cognitive disorder among recipients and bystanders to a greater or lesser degree. This is where some form of “sense-giving” or symbolic management is desperately needed. Clear and credible visions about the importance and value of change and where change is heading will be helpful, as will investment in communications. Rather than enforcing rigidity around a guiding vision, shared brainstorming and networking, where potential leaders thrash out the meaning of the change and come to some joint negotiated understandings of how to move forward, may best support a coalition favourable to change. For change to become internalized within the organization’s culture, those involved in it must – perhaps paradoxically – have a certain flexibility to appropriate it in a way that makes sense for them. This process of negotiating the meaning of change might be supported by discussions on existing evidence, including information on the performance of current modes of health care delivery. In addition, to maintain the credibility of their visions, leaders need to pay close attention to the consistency of their own actions. For example, the credibility of the Quebec reform was not helped by ongoing and unannounced budget cuts that essentially cut into the promised reinvestment of resources into the health care system, which in turn undermined commitments to substantive change. Again, one of the most common errors made in change attempts involves skimping on the resource investments needed to support and sustain change. Indeed, an overall paradox underlies the whole notion of sustainability and of change in general. On the one hand, there is a need to ensure that desirable changes are formalized, embedded, and internalized. On the other hand, there is also a need to avoid ossification or rigidity. The notion of sustainability thus also implies continued adaptation and flexibility. Some have suggested that a conception of organizations as “complex adaptive systems” (Zimmerman et al. 2001), in which broad overarching visions combined with a small

102

The Curriculum Elements

number of simple guidelines serve to orient local adjustments, may be a better model for thinking about change than rigid, formally planned command and control structures, which tend to fail as they confront resistance. There is a lot to be said for adopting the “complex adaptive system” model in the context of health care organizations, where many people with different values, power, and interests interact to produce outcomes. Thus, although scientific evidence and planning may provide a basis for renewal, it is critical for health care managers to understand and manage the situated political and symbolic dynamics of change that will determine whether or not they will succeed and to be prepared to accommodate adjustments that fit within their overarching vision and objectives of health care improvement. In recent work, Canadian researchers Ross Baker and colleagues (2008) have studied a set of internationally known high-performing health care systems that have succeeded in moving their organizations towards a model in which high-quality care and patient safety are embedded in a continuous improvement culture – the very target that Sonja Glass and her colleagues were and are pursuing at Grey Bruce Health Services. Bate et al. (2008) conducted a similar study among US and UK hospitals that had outstanding records of organizing for quality improvement. Although somewhat different in focus from the case studies presented in this chapter (they involve intra-organizational changes, rather than broader system-level changes), these studies provide further evidence of the processes through which constructive transformations in health care can emerge and become institutionalized. The case studies show, as Sonja indicated in her introduction, that although the targets of high performance and adaptive health care may be the same in all cases, there is no “one best way” cookie-cutter approach to achieving these targets, nor are the resultant changes instantaneous or sporadic. Each change initiative needs to be situated within its specific context and requires continuous and persistent effort over long periods. In addition, rational, political, and symbolic management are clearly inherent to all these cases. These two recent studies of quality improvement, as well as other contemporary research on the achievement of sustainable health care change (e.g., Buchanan et al. 2006), will provide further inspiration for current and future EXTRA fellows and all those interested in evidence-informed health care change. In conclusion, research evidence on organizational change processes shows that rational argument in support of a given change initiative is

Managing and Sustaining Change

103

not enough on its own to support its implementation and institutionalization (Thomas et al. 1992) Aspiring leaders in health care need to pay attention to the context and complexity of change and develop complementary skills in political and symbolic management.

References Allaire, Y., and E.M. Firsirotu. 1985. How to implement radical strategies in large organizations. Sloan Management Review 26(3): 19–24. Armenakis, A.A., and A.G. Bedeian. 1999. Organizational change: A review of theory and research in the 1990s. Journal of Management 25(3): 293–315. Baker, G.R., A. MacIntosh-Murray, C. Porcellato, L. Dionne, K. Stelmacovich, and K. Born. 2008. High performing healthcare systems: Quality by design. Toronto: Longwoods Publishing. Balogun, J., and G. Johnson, 2005. From intended strategies to unintended outcomes: The impact of change recipient sense-making. Organization Studies, vol. 26 (11): 1573–1601. Bate, P., P. Mendel, and G. Robert. 2008. Organizing for quality: The improvement journeys of leading hospitals in Europe and the United States. Oxford: Radcliffe Publishing. Berwick, D. 2003. Disseminating innovations in health care. Journal of the American Medical Association 289:1969–75. Buchanan, D., and R. Badham. 1999. Power, politics, and organizational change. Thousand Oaks, ca: Sage. Buchanan, D., L. Fitzgerald, and D. Ketley. 2006. The sustainability and spread of organizational change: Modernizing Healthcare. London: Routledge. Denis, J.-L., A. Langley, and L. Rouleau. 2006. The power of numbers in strategizing. Strategic Organization 4(4): 349–77. Doran, T., C. Fullwood, E. Kontopantelis, and D. Reeves. 2008. Effect of financial incentives on inequalities in the delivery of primary clinical care in England: Analysis of clinical activity indicators for the quality and outcomes framework. The Lancet 372(9640): 728–36. Dunphy, D.C., and D.A. Stace. 1988. Transformational and coercive strategies for planned organizational change: Beyond the OD model. Organization Studies 9(3): 339–55. Golden-Biddle, K., and T. Reay. 2009. Negotiating a new role in a gendered order: A cultural lens. Journal of Negotiation and Conflict Management Research 2(1): 31–41.

104

The Curriculum Elements

Ham, C. 2003. Improving the performance of health services: The role of clinical leadership. The Lancet 361(9373): 1978–80. Huy, Q.N. 2001. Time, temporal capability and planned change. Academy of Management Review 26(4): 601–23. Kerr, S. 1975. On the folly of rewarding A while hoping for B. Academy of Management Journal 18(4): 769–83. Lawrence, T.B., M.K. Winn, and P.D. Jennings. 2001. The temporal dynamics of institutionalization. Academy of Management Review 26(4): 624–44. Reay, T., K. Golden-Biddle, and K. Germann. 2006. Legitimizing a new role: Small wins and micro-processes of change. Academy of Management Journal 49(5): 977–98. Rodriguez, R., A. Langley, F. Béland, and J.-L. Denis. 2007. Governance, power and mandated collaboration in an inter-organizational network. Administration and Society 39(2): 150–93. Scholes, K. 2001. Stakeholder mapping: A practical tool for public sector managers. In Exploring Public Sector Strategy, edited by G. Johnson and K. Scholes, 165–84. Pearson Education: Prentice-Hall. Shediac-Rizkallah, M.C., and L.R. Bone. 1998. Planning for the sustainability of community health programs: Conceptual frameworks and future directions for research, practice and policy. Health Education Research 13(1): 87–108. Thomas, J.B., D.J. Ketchen, L.K. Trevino, and R.R. McDaniel. 1992. Developing interorganizational relationships in the health sector: A multi-case study. Health Care Management Review 17(2): 7–19. Vaughn, T., M. Koepke, C. Kroch, et al. 2006. Engagement of leadership in quality improvement initiatives: Executive quality improvement survey results. Journal of Patient Safety 2(1): 2–9. Weick, K.E., and R.E. Quinn. 1999. Organizational change and development. Annual Review of Psychology 50: 361–86. Zimmerman, B., C. Lindberg, and P.L. Plsek. 2001. Edgeware: Insights from complexity science for health care leaders. Dallas, TX: VHA Inc.

8 Mentoring Adults for Research Application samuel b . sheps

Mentoring is a brain to pick, an ear to listen, and a push in the right direction. John Crosby

Prologue As a student, I was offered the opportunity to choose a mentor to guide me through my medical school years. I remember the vague uneasiness I felt at the time to be asked to select someone I did not know from a list of volunteer faculty members. In the end, I elected to pass up the opportunity, preferring instead to ask a few teachers whom I had encountered during my studies and whom I respected to help and support me as I went through my challenging learning adventure. In fact, I and several others did end up establishing wonderful relationships with talented individuals who shaped our thinking and our experiences in the art and science of medicine. These were the real mentors: they had the passion, the talent and the open-mindedness to nurture and shape us into what we would ultimately become. Mentoring truly is the art of creating excellence in others. What do mentors need to do to become good mentors? To develop the special relationship created by mentoring, mentors must above all respect their protégés. This is crucial; it ensures that the protégé has the right to make mistakes and to learn from them. Mentors have experience and knowledge, and they must be ready to share all of it. Differences in ages do not matter – it is the spirit and the willingness to engage in a dialogue that counts. Mentors can become enthusiastic

106

The Curriculum Elements

over something the protégés have developed and can guide them through an original project while learning from their students along the way. Mentors are available, not just for face-to-face meetings but also to answer questions as issues arise. And with today’s electronic tools and global society, the mentor/protégé relationship can be maintained even from one end of the world to the other. On the other hand, protégés must choose the right mentors; there needs to be complicity between the teacher and the student. This is particularly the case in the EXTRA program, where fellows have the opportunity to work with up to three mentors. For most of us in the EXTRA program, having a mentor with leadership knowledge and change management skills was essential to our learning. While it is not always easy to find such people, the expertise of former fellows and counselling from the regional mentor centres proved to be very helpful. Choosing an academic mentor can prove to be even more challenging if the fellow is not in an academic institution. What can EXTRA fellows expect to experience when they are twinned with mentors? First, there is an opportunity to seek affinities, “des atomes crochus” (a picturesque expression used to describe a natural affinity towards another individual), and to define and appreciate each other’s role. As this unfolds, a mutual trust emerges and a feeling of confidence starts permeating the relationship, allowing the fellow’s work to progress and supporting him or her in the process. The first two modules of the EXTRA program clearly test the solidity of the relationship, since they are geared towards increasing the fellows’ knowledge base. The number of papers and other documents to review can be daunting. Mentors help in managing the influx of information and in ensuring that we keep focused on the end goal: the intervention project. An interesting complementary process occurs during the modules, when members of the EXTRA faculty also become mentors and are available for one-on-one sessions with fellows to discuss any aspect of the intervention project. As the EXTRA program unfolds, fellows realize that they are recipients of a great gift: superb teaching, coaching, and mentoring. This support is available throughout the two-year program. My colleagues, Madeleine Boulay Bolduc and Patricia Lefebvre, and I were the first group to apply and to be accepted as a team in the EXTRA program. In addition to managing our course work and projects, we faced the added expectation of demonstrating that teams could successfully complete the program. The choice of mentors was a critical step for us, and we were truly blessed with

Mentoring Adults for Research Application

107

the result! We are and will be forever grateful to our mentors for their exceptional support throughout our EXTRA program. They enriched the experience in so many ways because of their ability to support us, coach us, participate in our successes, and help us find solutions to the many challenges we had, and will likely continue to have, to make our intervention project an ongoing success. Our gratitude extends to all our teachers during this exciting and productive two years. May all future fellows have this enriching and lasting experience! Micheline Ste-Marie, 2009 Graduate

Introduction Mentoring is an important part of the Executive Training for Research Applications (EXTRA) program. However, the program’s rationale for, and approach to, the concept of mentoring varies significantly from traditional practice. The EXTRA fellows’ tasks are many and diverse. All of these tasks are undertaken over eighteen months. The fellows’ individual learning, as well as changes within their organization, creates an intense personal and professional dynamic that requires considerable support. Since the fellows are grouped into cohorts of twenty-four to twenty-eight individuals and since two cohorts concurrently engage in the formal learning residency sessions, considerable peer interaction is fostered. However, early in the development of the EXTRA program, it was felt that fellows faced a significant challenge, particularly in the period between the residency sessions, in maintaining the momentum required to clarify the problem they had defined in their organizations and in initiating and maintaining the identified solution. Addressing this challenge from an organizational perspective is the responsibility of the sponsor, whose primary role is to create and maintain support within the organization for the fellow and his or her intervention project. However, the fundamental rationale for establishing an EXTRA mentoring structure and process was to provide ongoing individual and organizationally independent support for the fellows between residency sessions.

Mentoring: A Brief Overview Traditionally, mentoring has been conceptualized as a form of supportive relationship between a senior mentor and a junior mentee, or

108

The Curriculum Elements

protégé. Early research conceptualized mentoring as having two key functions: career advancement and psychosocial support (Kram 1983, 1985). More recent research has added the components of job satisfaction, the protégé’s commitment to his or her organization, personal competence, and enhanced retention as critical outcomes (Allen et al. 2004; Noe et al. 2002; Wanberg et al. 2003). As well, there have been debates regarding the relative merits of informal and formal mentoring (Ragins and Cotton 1991, 1999). However, as noted by Wanberg et al. (2003), until recently most research on mentoring has failed to distinguish between formal mentoring, in which the mentor and protégé are explicitly matched, and informal mentoring, in which they find each other through personal networks. Research has also not taken into account the length and intensity of the mentoring process, which, as noted by a number of authors, varies widely (Chao et al. 1992; Noe 1988). The direct relationship between mentoring and any of the number of outcomes noted above has been questioned, since the literature contains studies both supporting and not supporting the strength of these relationships (Ragins and Cotton 1999; Russell and Adams 1997). The key issue, aside from design limitations, is that confounders of successful mentoring have not been adequately controlled. (Most studies of mentoring until very recently have been cross-sectional, so causal relationships over time cannot be measured.) This raises the possibility that early research reported spurious associations (Kammeyer-Mueller and Judge 2008). Nevertheless, in their synthesis of the literature these authors report that when personal attributes of protégés and mentors, as well as organizational characteristics, are controlled, mentoring does have important effects on career and job satisfaction. However, self-evaluation and education of protégés have stronger effects on salary, advancement, and other career objectives and outcomes. Similarly, Egan and Song (2008) used a pre-test/post–test randomized field experiment among employees in a large Fortune 500 health care organization. Participants in each group were compared on a number of key demographic and pre-test measures comparing highlevel (formal) and low-level (informal) facilitated mentoring programs to a control group. While both mentored groups showed increases in job satisfaction, organizational commitment, personorganization fit, and manager-rated performance compared to controls, the improvement in the outcomes was greater in the high-level facilitated group.

Mentoring Adults for Research Application

109

One of the more interesting aspects of recent mentoring research is the exploration of cultural differences. Clutterbuck (2007) notes that conceptions of mentoring are very different in Europe as compared to North America. The primary difference was that northern European mentoring emphasized personal development and learning, while North American mentoring focused on career outcomes and the protégé’s advancement. These are qualitatively quite different outcomes, and the European approach is much more aligned with the objectives of the EXTRA program. Another interesting aspect of the more recent literature is the relation of mentoring to leadership. Godshalk and Sosik (2007) discuss transformational leadership behaviours as significantly overlapping with the qualities of a good mentor. While this literature has yet to demonstrate a clear relationship between leadership and high-quality mentoring, a number of authors note that leadership tends to focus on organizational change, while mentoring focuses on individual change (Kouzes and Posner 2003; McCauley and Guthrie 2007; Ting and Scisco 2006). Given that EXTRA mentoring is designed to support both organizational and personal change, the development of leadership skills is central to its objectives. This makes EXTRA mentoring substantively different from traditional mentoring dynamics, supporting the fellows in their intervention project work and in their personal growth. Moreover, leadership and change management competencies are key components of the EXTRA curriculum. The unique focus on organizational change through the intervention project aligns the broader mentoring and leadership objectives of the EXTRA program. One aspect curiously omitted from the mentoring literature is what kind of people are appropriate mentors. The psychological attributes of good mentors have been extensively discussed, although not from a disciplinary perspective. It is generally found that mentors are senior leaders with long experience in the organization and that they convey explicit knowledge and experience of leadership. They also convey, implicitly and explicitly, the organizational culture to the protégé. Traditional mentoring is generally bounded by discipline, and mentoring research can be characterized as conceptually one-sided given the assumption that the mentor and mentee would naturally have very similar, if not identical, disciplinary backgrounds. Cross-disciplinary mentoring is not highlighted in the mentoring research.

110

The Curriculum Elements

Implications of the Literature on Designing the EXTRA Mentoring Process The characteristics of the EXTRA fellows as experienced and senior health care managers suggests that the traditional hierarchical dynamic, in the sense of the guidance and development of a neophyte, was not aligned with the objectives of the EXTRA program. In addition, there were dual challenges with the program: first, to foster leadership within the context of evidence-informed change management and, second, to deliver explicit training on finding and using research evidence and integrating that evidence with other forms of institutional evidence. These factors suggested that mentoring was needed not only to support organizational change but also for finding, understanding the messages of, and integrating research evidence into the decision-making process. In response to these challenges, two significant departures from traditional mentoring were incorporated into the EXTRA program. The first was to adopt a collegial, peer-to-peer mentoring relationship aligned with recent mentoring research (Ragins and Kram 2007) and focusing on developmental issues, particularly with regard to leadership. This approach recognized specifically that the traditional mentoring model needed to be adapted to make it more relevant to senior protégés, an observation made (but interestingly, not given much attention in the literature) two decades ago (Wilson and Elman 1990). The second departure was the importance placed on academic mentoring. This was based on the need to build fellows’ competencies in the identification, synthesis, and communication of key messages derived from research and other evidence. The experience emerging from this new approach has highlighted the importance of voluntary choice, as opposed to assignment, for pairing the mentors and fellows. It also highlights the need for the fellows to take primary responsibility for managing the dual (and more complex) mentoring relationship through regular and planned interaction, setting a clear agenda and expectations, following through on agreed upon action (Hamlin and Sage 2008), and responding to the critical need for face time as opposed to distance communication. It was recognized that because research-evidence seeking and application were new concepts to many decision makers, explicit support for these processes was warranted.

Mentoring Adults for Research Application

111

EXTRA Mentoring Given the dual facet of the EXTRA curriculum, the basic structure of EXTRA mentoring support includes two mentors: one academic and one decision-maker. A regional structure that facilitated and supported the identification of relevant and experienced mentors and that provided ongoing assistance – for example, trouble-shooting – also supported the mentoring process.

A Tale of Two Mentors As noted above, a significant component of the EXTRA curriculum, as well as a key objective of the intervention project, is the application of research-based and other evidence in identifying problems and assessing solutions. Fellows, many of whom have had little research training or experience, are taught skills related to finding research evidence, synthesizing and integrating it with data from their own organizations, and understanding aspects of research design. (See chapter 5 for details.) It was recognized early in the development of the EXTRA program that explicit individualized support for these activities in between the time of the residency sessions was critical to sustaining fellows’ progress with their intervention project. While the role of a decision-maker mentor was considered axiomatic, equally clear was the importance for fellows to have a mentoring resource focused on the research aspects of the intervention project, in order to reinforce the research components of the curriculum. While not every fellow has had two mentors (some have found individuals with both management and research experience), most have, and this flexibility has been a particular strength of the program (see below). Several other aspects of EXTRA mentoring should be noted. First, the decision-maker mentor did not have to come from the fellow’s organization, which meant that the range of potential mentors was significantly widened. Choosing a mentor outside of, or in a very different part of, the organization would allow the freedom to discuss difficult aspects of the organizational culture, leadership, or staff issues without either fear of sanction or subtle efforts to make the fellow conform to organizational norms. It was also felt that fellow-mentor engagement should be tangibly supported (mentors

112

The Curriculum Elements

are paid a small honorarium), and be made administratively clear through the use of a mentoring agreement that sets forth the responsibilities of the mentor and the fellow. There is little if any precedent in the mentoring literature for these aspects.

Regionalization The EXTRA program inducted its first cohort of fellows in the spring of 2004. Three years before, the CHSRF had established, based on a peer-reviewed selection process, three university-based regional training centres (RTCs), adding a fourth centre in 2002, to support graduate inter-disciplinary educational programs in health services research. This meant that by the time the EXTRA program began, there was an infrastructure focused on health services research based at four universities in Atlantic Canada (ARTC) at the University of New Brunswick, the University of Prince Edward Island, Dalhousie University, and Memorial University; in Quebec (Centre FERASI) at the Universities of Montreal, Laval, and Sherbrooke; in Ontario at six universities (OTC); and in the four western provinces (WRTC) at two universities, the University of British Columbia and the University of Manitoba (and since 2007, at three, with the addition of the University of Alberta). Given the need for ongoing support for fellows and given that EXTRA is a pan-Canadian program, it was thought that the regional training centres – using their existing resources – could provide a leadership role in identifying regional academic and decision-maker mentors. Thus, the RTCs became the Regional Mentoring Centres (RMCs) for the EXTRA program. Their primary responsibilities have been to assist fellows in finding both academic and decisionmaker mentors to monitor the ongoing relationship and provide support if problems arise and to evaluate the quality of the mentoring relationships. From a program perspective, a key element enhancing both the regional role for mentoring support and a consistent program-wide approach has been the monthly RMC teleconference. This regular communication, which fosters mutual learning with regard to the mentoring experiences across the program and between cohorts, assists in implementing new program activities, such as curricular changes, that may affect mentoring. Recently, RMCs accepted the responsibility of identifying an individual from the region to take on the organizational liaison component

Mentoring Adults for Research Application

113

of the EXTRA program. The individual who was the organizational liaison for the entire country is a respected and experienced health care administrator and has now moved into the role of organizational liaison for Ontario and Quebec. The West and Atlantic Canada have selected individuals of equal stature to take on this role in their respective regions. RMCs have taken on other responsibilities that, while directly related to mentoring, also help to support the EXTRA program more generally. For example, they organize EXTRA program orientation sessions for new fellows before the first summer residency session. A key feature of the orientation is to emphasize the importance and rationale of the mentoring component of the program and to introduce all facets of the program, including an initial orientation to the Desktop (see chapter 10). The orientation includes fellows from previous cohorts, as well as, in some cases, individuals who have mentored previous cohort fellows. Recently the regional organizational liaison individuals have become key participants, allowing them to get a sense of their fellows and their organizations before visiting each new fellow and their organization sponsors. A mentoring lunch at the end of the first residency session helps to create synergies between the two cohorts and stimulate dialogue on mentoring for mutual learning and application. The RMCs play a key role in introducing new fellows and sponsors (some of whom are new and some of whom are not) to the overall objectives of the program and key elements of its structure and processes. They also serve to re-emphasize the importance of mentoring – both academic and decision-making mentoring – for providing sustained support for fellows throughout the definition and implementation of the intervention project. Moreover, the regional approach provides an enhanced contextual dimension of the  mentoring concept, which has been increasingly regarded as important in mentoring research (Ragins and Kram 2007; Kilcher and Skretis 2003).

Change Management within the EXTRA Program One of the key attributes of the EXTRA program has been its ability to adapt to the mentoring needs of differing cohorts based on their feedback, while maintaining a core structure of goals and activities. An example of the capability for reflective change informed by

114

The Curriculum Elements

fellows’ and faculty experience and by the formal evaluation of mentoring specifically (see below) was the replacement of the programwide mentoring coordinator with a regional focus on mentoring linked to an organizational liaison role. These changes reflected the need for more direct feedback on the intervention project reports and a clearer recognition of the close linkage between mentoring and organizational support. It was felt that regional resources were better able to facilitate communications between fellows and mentors, which, in turn, helped to establish the kind of relationships that reflect successful dynamics of mentoring. Another recent addition to the mentoring component has been the designation of regional reviewers of the fellows intervention projects (IPPRs) who are responsible for providing fellows and their mentors with an independent review of their IPPR at two different points in the evolution of their intervention project. The reviewers not only provide comments on the electronic IPPR (using tracked changes) but also complete a form summarizing their comments and impressions of the IPPR. A small honorarium is paid to the reviewers. No formal feedback on the merit of this review process, particularly its utility in relation to the overall mentoring process (complementary or redundant) has yet been undertaken.

Evaluation Early in the EXTRA program two forms of evaluation were established. One was a simple screening assessment tool, the Mentoring Performance Assessment (MPA), which used a four-point scale – poor, insufficient, sufficient, excellent. Three times during the program it was sent to fellows and to academic and decision-maker mentors to assess the mentoring process for each fellow. Feedback was solicited along several dimensions: time invested by the mentor, support and advice with regard to the intervention project, communication, clarity of expectations regarding the mentoring role, support from the regional mentoring centre, and support from the organizational sponsor. The screening assessment tool was designed to give RMCs an early warning of mentoring issues (lack of support, poor communications) so that they could understand the nature of the problems arising and intervene with solutions as needed, solutions that could include, in extreme situations, finding new mentors for the fellows.

Mentoring Adults for Research Application

115

In addition, narrative comments were solicited about the ratings, as well as feedback on what was working well and recommendations for improvement. As will be noted in the next chapter (9), on evaluation, the mentoring process has improved significantly, particularly with regard to the role of the RMCs across the four cohorts whose screening data are available. Our interpretation of this trend is related to an increasing comfort that the fellows and RMCs have with the process and the enhanced role – specifically, the increased standardization of the orientation – of the RMCs in the EXTRA program overall. The recent change to a regional role for the organizational liaison has been well received. It is clear from early experience that this change has been very helpful in quickly addressing fellows’ needs as their environments change or as they change positions, by bringing regional knowledge regarding the health care landscape and a known face to the table and by providing an additional regional perspective to the monthly RMC teleconferences. For example, in the West, through his connections the organizational liaison was well aware of the impending changes to the Alberta regional health structure. He was able to sense the implications of this change very shortly after the announcement was made and has served as a knowledgeable observer who can monitor subsequent events in Alberta as the new structure unfolds. This has been very helpful to an Alberta fellow, whose job had suddenly and radically changed. A more qualitative perspective is provided by fellows’ responses to questions from the EXTRA evaluation team. Each cohort was surveyed in their first residency session about their expectations: first, “What are the main things you hope to receive from the EXTRA mentoring?” and second, “How has the mentoring experience been a positive experience for you?” (Despite the positive spin of the second question, some negative responses were elicited.) With regard to the first question the fellows answered: •

• • • •

Someone to help me focus/clarify my thinking processes & project plan. Getting connected to experts to support by project. Encouragement to “think outside the box.” Assistance with issues related to helping change my organization. Still very unclear as to the role of the mentoring program in EXTRA … To date I have sought out two mentors that I feel can support me as I move my intervention project forward.

116

The Curriculum Elements

At this early stage in the EXTRA program the fellows’ expectations about mentoring were still hazy, although some clearly understood what mentoring was all about. It should be noted that the first cohort did not get the pre-residency session orientation described above about mentoring or the role of mentoring centres (in fact, the RMCs didn’t exist at that point). For this first cohort answers to the second question revealed some continuing uncertainty: •

•

•

Disappointed that the provincial group [RMC] never did provide a list of possible mentors and their expertise. I recommend this for next year. Not a clear understanding of the role of the two mentors (academic and administrative). Unsure of what the role of the regional mentoring centre really is, but I understand this will probably be worked out eventually.

However, there were also positive opinions about mentoring that illustrate its multi-dimensional character. •

• •

•

Although the academic mentorship has been slow to get off the ground I have found that my mentor has provided good advice re: frameworks, approaches and structure for my project. A very effective stimulus to engage into a proactive leadership role. A good insight on the common challenges health leaders are facing every day. Opportunity to spend time with and learn from others who are more experienced.

For subsequent cohorts the mentoring experience was generally more positive, although fellows in cohort 3 noted that “it would be a stronger experience if mentors had a better understanding of the deliverables of the EXTRA program and the IP in order to ensure alignment between what they can offer and the expectations/requirements.” This and other similar comments led to the more formal orientation session and an enhanced role for the RMCs, extending their role beyond identifying mentors to a more active discussion about all aspects of the EXTRA program expectations and deliverables. Another issue that arose early in the program was a concern about an over-emphasis on the academic expectations: “The quality and availability of the mentoring has been terrific. I think sometimes my mentors forget the academic writing/research is not my expertise

Mentoring Adults for Research Application

117

and can be a little bit overly critical of my IP.” This comment suggests a clash of cultures – academic vs decision maker – that is in some sense intrinsic to the nature of the program and reflects, as well as addresses, the broader tension between these two worlds. However, other comments from later cohorts include: •

•

•

•

•

•

•

•

Has provided me with a new skill set and a way to approach and plan meaningful change. Presents the opportunity to regularly discuss my project and the more global purpose of EXTRA with key people (mentors) and receiving coaching, feedback, and advice. It is the proverbial “stretch learning” experience! I feel quite inadequate most of the time; however, I see very specific categories of skills emerging. Academic mentor very helpful in assisting me to define problem, frame IPPR [Intervention Project Progress Report] and helpful feedback; Org decision maker assists in communication within organization. Regional mentoring centre … has been a great resource; Academic mentor of little utility; Decision making mentor has been useful in furthering the project. Gained important insight in research and use of evidence … mentor an expert in the field related to my IP and major organizational responsibilities. I have two very bright mentors and they have given me very good advice on moving the project forward, ensuring I don’t allow scope creep. I am benefiting from the advice and guidance of my academic and organization mentors who are helping me to frame my project, suggesting issues for my consideration and integration.

These comments suggest that the mentoring process is working as designed and has improved, though they reflect more of a formative than a summative evaluation. However, until a more comprehensive publication of these evaluations provides a longitudinal description of impact, it can be said that the fellows, by and large, feel supported in undertaking their intervention projects, that the balance between academic and decision-maker mentors works for most and that there has been increasing clarification of the mentoring role. As noted, a hallmark of the EXTRA program has been a continued evolution in response to fellows’ views, the program being adjusted as

118

The Curriculum Elements

the fellows’ needs dictate yet retaining its basic structure and processes. The EXTRA program represents the best type of educational initiative; it is organic and alive.

Cost of Mentoring No discussion of the EXTRA mentoring component would be complete without a discussion of the resources needed to develop and run the type of mentoring described above. The major costs are support for the Regional Mentoring Centres (which does not include any salary support for the RMC director), primarily to provide assistance to the RMC director; payment of an honorarium to the mentors themselves; payment for regional IPPR reviewers, who in some cases are the RMC directors; contracting with the regional organizational liaison; travel; and communications. The RMCs are paid $15,000 annually, and the mentors are paid a nominal honorarium of $1,000 a year, a sum far below what they could command as private consultants and for which they have made significant commitments in time and energy to support the fellows. The organizational liaison budget is provided to the RMC to a level of $5,100 a year, and the honorarium for each regional site is variable and negotiated between the RMC and organizational liaison. Travel costs are paid in part by the RMC (for the orientation sessions and the mentoring lunch at the end of the first residency module) and in part by the CHSRF directly, for organizational visits. The regional IPPR reviewers are paid $500 a year to review and comment on IPPRs 1 and 2.

Conclusions The EXTRA program is unique. It is not an educational program based in the academy, nor is it an apprenticeship program. Competencies and skills conveyed to EXTRA fellows are focused on specifics (understanding key features of research methodology), and they are conceptual (integrating differing kinds of evidence and what can be learned or understood from organizational and social and professional contexts, leadership and change management) so that fellows can achieve significant organizational change through their intervention projects. Such experiential learning, grounded in an explicit problem-solving process (the IP), requires support of a far more varied nature than the support required for simple skill-building or

Mentoring Adults for Research Application

119

career enhancement. The EXTRA fellows are not novices; however, though they may be experienced professional decision makers and managers, they are, like all individuals, wedded to an organization and therefore institutionalized both specifically in their individual organizations and in health care more generally. The EXTRA program in general, and mentoring specifically, is meant to facilitate transformational change in ways different from traditional mentoring programs; it aims, among others things, to liberate the fellows from the institutionalization that has accreted around them. It does not just create skills; rather, it molds skills and knowledge already developed but constrained by the almost self-imposed limitations that organizational cultures exert. It uses the experience of the fellows in novel ways and for a very explicit purpose: to solve organization problems and through interaction with their peers and members of the academy, to learn, as one fellow put it, “to think outside the box about what is going on inside the box.” The mentoring component is flexible, yet structured, and explicitly addresses both the organizational support needs of the fellows and a specific range of competencies in areas such as leadership and research assessment and integration. Moreover, the mentoring components of the program are critical to supporting wider organizational learning and change, the achievement of which is a core objective of the program.

References Allen, T.D., L.T. Eby, M.L. Poteet, E. Lentz, and L. Lima. 2004. Career benefits associated with mentoring for proteges: A meta-analysis. Journal Applied Psychology 89:127–36. Chao, G.T., P. Walz, and P.D. Gardner. 1992. Formal and informal mentorships: A comparison of mentoring functions and contrast with nonmentored counterparts. Personnel Psychology 45:619–36. Clutterbuck, D. 2007. An international perspective on mentoring. In The Handbook of Mentoring at Work: Theory, Research and Practice, edited by B.R. Ragins and K.E. Kram, 633–55. Thousand Oaks, Ca: Sage Publications. Egan, T.M., and Z. Song. 2008. Are facilitated mentoring programs beneficial? A randomized experimental field study. Journal of Vocational Behavior 72:351–62.

120

The Curriculum Elements

Godshalk, V.M., and J.J. Sosik. 2007. Mentoring and leadership: Standing at the crossoads of theory, research and practice. In The handbook of mentoring at work: Theory, research and practice, edited by B.R. Ragins and K.E. Kram, 149–78. Thousand Oaks, CA: Sage Publications. Hamlin, R.G., and L. Sage. 2008. Behavioural criteria of mentoring effectiveness: An empirical study of formal mentoring relationships within a major UK public sector organization. The Work Foundation Working Paper. Kammeyer-Mueller, J.D., and T.A. Judge. 2008. A quantitative review of mentoring research: Test of a model. Journal of Vocational Behavior 72:269–83. Kilcher, A., and I. Sketris. 2003. Mentoring resource book: A guide for faculty, researchers and decision makers. College of Pharmacy, Dalhousie University. Canadian Pharmacists Association Graphic Communications, Ottawa Ontario. Kouzes, J.M., and B.Z. Posner. 2003. The leadership challenge. San Francisco: Jossey-Bass. Kram, K.E. 1983. Phases of the mentor relationship. The Academy of Management Journal 26:608–25. – 1985. Mentoring at work: Developmental relationships in organizational life. Glenview, IL: Scott Forseman. McCauley, C.D., and V.A. Guthrie. 2007. Designing relationships for learning into leader development programs. In The handbook of mentoring at work: Theory, research and practice, edited by B.R. Ragins and K.E. Kram, 573–91. Noe, R.A. 1988. An investigation of the determinants of successful assigned mentoring relationships. Personnel Psychology 41:457–79. Noe, R.A., D.B. Greenberger, and S. Wang. 2002. Mentoring: What we know and where we might go. Research in Personnel and Human Resources Management 21:129–73. Ragins, B.R., and J.L. Cotton. 1991. Easier said than done: Gender differences in perceived barriers to gaining a mentor. Academy of Management Journal 34:939–51. – 1999. Mentor functions and outcomes: A comparison of men and women in formal and informal mentoring relationships. Journal of Applied Psychology 84:529–50. Ragins, B.R., and K.E. Kram, eds. 2007. Handbook of mentoring at work: Theory, research and practice. Thousand Oaks, CA: Sage Publications. Russell, J.E.A., and D.M. Adams. 1997. The changing nature of mentoring in organizations: An introduction to the special issue on mentoring in organizations. Journal of Vocational Behavior 51:1–14.

Mentoring Adults for Research Application

121

Ting, S., and P. Scisco, eds. 2006. The ccl Handbook of coaching: A guide for the leader coach. San Francisco: Jossey-Bass. Wanberg, C.R., E.T. Welsh, and S.A. Hezlett. 2003. Mentoring Research: A review and dynamic process model. Research in Personnel and Human Resource Management 22:39–124. Wilson, J.A., and N.S. Elman. 1990. Organizational benefits of mentoring. Academy of Management Executive 4:88–93.

9 Evaluating the First Years of the extra / forces Program malcolm anderson , melanie lavoie - tremblay, lynda atack , dorothy forbes , susan donaldson , manon lemonde , lorna rornilly, lyn shulha , ingrid sketris , richard thornley, and stephen tomblin

Introduction In this chapter we present data on the first four years of the evaluation of the EXTRA/FORCES program. The evaluation has been based on a blend of utilization-focused evaluation (Patton 1997), responsive evaluation (Stake 2004), and theory-driven evaluation (Donaldson 2003). We begin the chapter by providing a short overview of the evaluation’s focus and some of the background literature (members of the evaluation team are listed in the appendix to this chapter). This is followed by a profile of the fellows in each cohort. We present a range of data on the various components of the program, including the residency sessions, the role of mentoring, knowledge uptake, the transfer of knowledge, and the intervention projects. The predominant and consistent evidence thus far is that the program is responsive and works well.

Context Understanding the organizational context in which change occurs has been informed by studies that broaden analysis beyond efficiency

Evaluating the First Years

123

and cost or economic factors associated with innovation to examine causal processes, cultural and structural factors, relationships with other organizations, and formal and informal networks around which social relationships lead to the exchange of tacit and explicit (codifiable) knowledge. The creation of a receptive organizational context for change is a major challenge for organizations (Anderson 2006). The work of Pettigrew et al. (1992) on receptive and non-receptive contexts for change, in particular, informs evaluative research. Three key constructs frame the successful development, implementation, and sustainability of innovations in the EXTRA host organizations: • • •

Context: the “why” and “when” of change Content: the “what” Process: the “how” and “by whom”

Process is a critical construct, especially in the context of the EXTRA program. The processes that enhance the use of research evidence in EXTRA host organizations are mediated by a range of approaches or strategies (modes) to ensure ongoing knowledge exchange. Each different mode of exchange is itself mediated by the temporal element (the time required for various changes to occur; Pettigrew et al. 1992), the complexity of the socialized work environment characterized by interactions based on exploratory, intuitive, and responsive social actions, and the perceived and committed role of evidence in supporting and enhancing the ongoing commitment to use research evidence to informing decision making. As the EXTRA program evolves, a greater understanding of the details underlying the context, content, and process constructs is of considerable benefit to fellows (and their organizations) subsequently entering the program and to other health care organizations that are interested in developing a more evidence-informed approach to decision making. Simply put, the research evidence application experiences of the fellows over the first few years of the program will enable us to develop a greater level of understanding of what is required for a receptive organizational context to use research evidence. With that detailed knowledge, we can then determine what supports organizations need for enhancing research evidence use, and what factors function to undermine or prevent evidence use.

124

The Curriculum Elements

The implications for resources are significant. With the increasing desire to see a return on investment for health research (and innovations) (Canadian Academy of Health Sciences 2009), it makes sense to acknowledge the heterogeneity of organizational capacity and to know when and what type of specific resources should be directed to which organizations, with what set of expected specific outcomes. Evaluative research on the receptive organizational environments benefiting from the EXTRA program can greatly inform this direction. Changes associated with EXTRA are occurring simultaneously with many other changes also occurring in what is now a constantly changing and increasingly complex decision-making environment. While it is absolutely vital that the overall final evaluation of the program incorporates an assessment of changes at the meso level and at the broader macro or health system level, the primary focus of the initial phase of evaluation has been at the micro level of the fellows themselves, in order to (a) confirm that expected changes are occurring with the fellows themselves (i.e., that the different cohorts are learning new knowledge and skills) and (b) in order to explore what changes have occurred and the various strategies and challenges that have emerged through the diverse experiences of the fellows. Change takes time and, in the context of the EXTRA fellowship, often more than the two-year fellowship period. We are now in a position, however, to build on the experiences of the fellows during and subsequent to their two-year fellowship to more fully confirm and understand what outcomes have been achieved and the causal mechanisms that have been used. We are currently developing several more in-depth research projects to further understand what changes (their content) have occurred, how and by whom (the process), and why and when (the context).

The Theory of Learning The theory of learning used by the program is centred on an adultlearning approach and an environment that is “responsive to the needs and aware of participants’ considerable existing skills and knowledge” (Design Working Group 2003). Adult learners are typically problem-centred and results-oriented and self-directed, that is, not dependent on others for direction. They can be skeptical about

Evaluating the First Years

125

new information and prefer trying it out before accepting it. Importantly, they seek educational avenues that relate directly to their perceived current needs.

Evaluation Design and Expected Outcomes The EXTRA program is designed to transfer knowledge to fellows and assist them in building capacity for using research evidence to inform decision making in their respective health care organizations. What happens in the fellows’ organizations during and following their fellowship period program is essentially a responsibility of the fellows themselves, the interaction with their mentors, and the degree of their engagement with their intervention projects. The changes that do occur, however, depend almost totally on the interplay of actions and beliefs of the fellows and the capacity, commitment, and interest of their organizations. While we have focused the evaluation to a large extent on the changes with the fellows, it is desirable and necessary to fully unpack the changes that do occur at the organizational level and, indeed, at the health system level. This is the focus of the current evaluation work. The evaluation is multi-layered, with outcomes thus far being examined predominantly at the micro individual level (e.g., changes observed with the fellows), and at the meso level (changes observed in culture and practice at the fellows’ organizations). Importantly, the more distal the outcomes, both temporally and as a function of the fellows sphere of influence, the more challenging it is to ascribe causal linkages to the EXTRA program itself. The evaluation has used multiple methodologies, which enable triangulation of the data to strengthen the validity of the findings. To date, methods employed include paper-based surveys of fellows (administered four times during their two-year fellowship), interviews with fellows, focus groups held annually with each second year cohort of fellows, review of administrative data from the CHSRF, review of evaluationrelated data from other program components (e.g., residency module evaluations, mentoring, and regional mentoring-centre evaluations), and reviews of the intervention project reports developed by the fellows. In 2005 the EXTRA/FORCES program theory was developed by the Evaluation Team and subsequently accepted by the CHSRF, the

126

The Curriculum Elements

Steering Committee, and the EXTRA/FORCES Advisory Council.1 The program theory states: Knowledge and skills acquired through residency periods, interresidency learning, mentoring, and networking and collaboration will be enhanced by web-based learning and supports and will result in improved knowledge transfer and uptake of evidencebased decision-making by individuals and organizations. This will lead to improved health care being provided across Canada. [emphasis added by the authors] Several major outcomes are expected from the program: •

•

•

•

Fellows acquire new skill-sets and knowledge from participating in the EXTRA/FORCES program. Fellows apply the skills learned and use research-based evidence to bring about organizational change The skills needed for improved use of research evidence in management are spread beyond those formally enrolled as fellows in EXTRA/FORCES. Fellows improve their capacity to collaborate as evidence-based decision makers between professional streams.

Kirkpatrick’s (1994, 1998) model for evaluating training effectiveness identified four levels of evaluation: •

• •

•

Reaction: responses of the participants (in this case the fellows) to the training. Did they like it, was it relevant, and so on? Learning: assessing the amount of learning that occurred; Transfer: how much of the learning, the new skills and knowledge, is being applied by the participants (the fellows)? Results: generically, what have the outcomes been?

The EXTRA/FORCES program is designed to transfer knowledge to the fellows, and they, in turn, transfer knowledge to their colleagues, thus helping to build capacity so that research evidence can be used 1 The theory is a synthesis of data collected from a review of all of the program’s developmental documentation and interviews conducted by the evaluation team with key stakeholders at the commencement of the program (i.e., CHSRF staff and Steering Committee members).

Evaluating the First Years

127

to inform decision making in health care organizations. Importantly, in the context of the evaluation, there is no direct EXTRA program connection with the host organizations aside from the training provided to the fellows, the fellows’ own development of their intervention project (IP), and an Impact Plan that the organization completes with the EXTRA Organizational Liaison. The focus of the program is heavily directed to the fellows, not the organizations, in terms of the time, training, education, and resources used.

Collecting the Relevant Data Although concerns may be expressed that the fellows themselves can provide only perception data, there is little else that can provide objective data specifically on the EXTRA program, unless considerable resources are directed to do so. There are also likely to be no other better sources of information on the EXTRA program’s effects inside an organization than the perspectives of the fellows. Even the reports on the intervention projects, which two of the evaluation team members have reviewed for the first three years (n=76), are prepared by the fellows themselves. The EXTRA program does not objectively test or examine the knowledge level of the fellows. Equally, it does not mandate changes in organizations. These two facts alone mean that there are no preexisting specified, standardized, quantitative, required benchmarks or expectations upon which to base an assessment of change in each organization. That does not mean that standardized assessments using existing instruments cannot be made; it does mean that they would need to fully reflect the module-based learning and content areas and would need to accommodate the highly variegated nature of the fellows themselves. There is much more involved in influencing change in organizations than achieving standardized scoring and developing benchmarks, and at this point in the program’s evolution there has not been a strong desire by the faculty to introduce such testing. The fellows are the best witnesses to any changes that do occur, because in the context of the EXTRA program, they are the conduits through which research evidence will be used to inform decision making in their organizations. Hence their views are critical to understanding the effects of the program. This is true in terms of their level of knowledge, their capacity to apply the knowledge, their understanding of what is happening in their own organizations, and

128

The Curriculum Elements

the future expectations they have – and others have of them – as a direct result of their participation in the fellowship.

Profile of the EXTRA Fellows As anticipated, a range of individuals have been involved in the program over the first five years. Most fellows (64 percent) are from Ontario or Quebec, reflecting the population of the country more generally. No applications have been submitted from the territories. Thirty-two percent of all applications submitted have come from regional health authorities and 27 percent from teaching hospitals. Each of these types of organization has provided approximately 30  percent of all fellows. Fifty-two percent of all applications submitted have come from health service executives, with another 31 percent from nurse executives and 17 percent from physician executives. This has led to 40 percent of all fellows being health service executives, 36 percent nurse executives, and 24 percent physician executives. Forty-three percent of all applications have come from directors, 15 percent from vice presidents and 11 percent from CEOs/executive directors. This has translated into 45 percent of all fellows being directors, 27 percent being vice presidents and 13 percent being CEOs/executive directors. The following tables present the profile data of the fellows in the program. There are no expectations for equitable distribution prevalent in the documentation. Simply, when the Advisory Panel reviews the applications they must first and foremost consider the quality of the candidates, regardless of their various positions and geographies. There is, however, a broad distribution of fellows across the categories, with the exception of representation from Northern Canada. There is also a small under-representation of physicians compared to the other two professional streams.

Do Fellows Acquire New Skill Sets and Knowledge? It is expected that the fellows will acquire new skill sets and knowledge to assist in the transfer of research evidence into the decisionmaking environment. For the most part this knowledge is shared through the residency sessions, which themselves are continually being realigned by the faculty and EXTRA (CHSRF) staff to better suit the needs of the fellows.

Evaluating the First Years

129

Table 9.1 Regional Distribution: Number of Fellows in the Program, Cohorts 1 to 5 Region

Total

Ontario Quebec

c1

c2

c3

c4

c5

Region

Total

c1

c2

c3

45

9

6

7

5

6

1

2

2

1

0

37

6

8

9

9

5 Saskatchewan

5

0

1

2

1

1

Nova Scotia

9

3

2

2

1

1 Newfoundland 4 & Labrador

1

0

0

3

0

Alberta

8

1

2

2

2

1 Prince Edward Island

1

1

0

0

0

0

New Brunswick

7

0

2

2

2

1 NWT, YK, Nun*

0

0

0

0

0

0

British Columbia

6

2

3

0

0

1

Total, 128 24 26 26 24 all regions

28

18 Manitoba

c4 c5

* Territories – Northwest Territories, Yukon, and Nunavut

Table 9.2 Number of Fellows in the Program by Organizational Type, Cohorts 1 to 5 Organizational Type Research hospital Regional health authority Community health centre Other Government ministry/department Long-term care facility Total, all regions

Total

c1

c2

c3

c4

c5

39 38 31 14 4 2

9 7 3 4 0 1

5 9 8 4 0 0

8 9 7 1 0 1

7 10 4 3 0 0

10 3 9 2 4 0

128

24

26

26

24

28

Table 9.3 Number of Fellows in the Program, by Leadership Stream, Cohorts 1 to 5 Leadership Stream

c1

c2

c3

c4

c5

Total

Health service executive

9

8

13

12

9

51

Nurses

9

10

9

8

10

46

Physicians

6

8

4

4

9

31

24

26

26

24

28

128

Total

130

The Curriculum Elements

Table 9.4 Number of Fellows in the Program, by Executive Positions, Cohorts 1 to 5 Executive Position Director Vice president CEO/Executive director Other Manager Coordinator Total

Total

c1

c2

c3

c4

c5

57 34 16 11 9 1

10 5 2 4 3 0

7 14 3 2 0 0

11 8 3 1 2 1

13 2 5 1 3 0

16 5 3 3 1 0

128

24

26

26

24

28

Source: Administrative data, EXTRA Program.

As part of the program evaluation, paper-based surveys are completed by fellows four times during their fellowship (baseline and at each of the three remaining residency sessions). A number of questions are repeatedly asked in each survey, and fellows are asked to answer many of them using a five-point Likert Scale, which typically goes from “excellent” to “poor.” Several questions focus on the fellows self-rating their knowledge and skill sets, the results for which are shown in table 9.5. As the results attest, there have been consistently high levels of satisfaction expressed about the residency sessions. Perhaps the main exceptions to this have been the views expressed by the first cohort, which should perhaps not be unexpected given the innovative nature of the program and the fact that the modules were being presented for the first time and expectations for the intervention project were clarified, as well as the roles of mentors. Data from questions asked repeatedly in each of the four paperbased surveys were analyzed to determine the extent of change that occurred in responses according to the fellows’ professional stream (nurse executives (NE), n=34; organization executives (OE), n=39; physician executives (PE), n=22), as opposed to each of the four specific cohorts. Where the sample size permitted, there were statistically significant changes observed in the survey responses for each of the professional streams between baseline (at first residency) and the fourth survey (which was administered in the fourth residency session for each cohort). The direction of the data for when there was insufficient sample size also strongly indicated that positive changes – expected improvements – were occurring. Figures 9.1 to 9.7 present these statistically significant changes.

Evaluating the First Years

131

Table 9.5 Overall Assessment of the Residency Modules by Fellows, Cohorts 1 to 4 Rated Excellent/Very Good (rated at the end of each module, percentages) Residency Module

Cohort 1 Cohort 2 Cohort 3 Cohort 4

Module 1: Demystifying the Research World (order switched in later cohort)

92

100

96

77

Module 2: Promoting the use of research-based evidence in healthcare organizations

74

100

79

96

Module 3: Becoming a leader in the use of research-based evidence

59

100

86

64

Module 4: Using research-based evidence to create and manage change

91

95

100

100

Module 5: Sustaining change in the organizational context

44

82

69

83

Module 6: Community of Practice, Presentation of Intervention Projects

72

88

70-86

71*

100

100

100

95

Improved capacity of fellows to apply research-based evidence

Source: Evaluation survey data, various years. * Denotes assessment of Intervention Projects (Community of Practice not a topic for Cohort 4).

Table 9.6 Overall Assessment of Skill Sets and Knowledge by Fellows, Cohorts 1 to 4 Overall Assessment of Skill Sets and Knowledge by Fellows

Percentage Change over Time (baseline to 4th residency) Cohort 1 Cohort 2 Cohort 3 Cohort 4

Knowledge of research evidence (excellent / very good)

60+

73+

53+

76+

Skill set for assessing quality of research evidence (excellent / very good)

25+

40+

33+

49+

Knowledge of change management (excellent / very good)

58+

45+

28+

42+

Ability to promote use of research evidence (excellent / very good)

65+

70+

55+

57+

Source: Evaluation survey data, various years.

132

The Curriculum Elements

Knowledge of research-based evidence

Amount of significant change in mean answer

1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0

ne

oe Executive Groups

pe

Figure 9.1 Change in knowledge of research evidence, by professional stream, cohorts 1 to 4

Use of research-based evidence in fellows’ work

Amount of significant change in mean answer

2 1.5 1 0.5 0

Figure 9.2 1 to 4

ne

oe Executive Groups

pe

Change in use of research evidence, by professional stream, cohorts

Evaluating the First Years

Amount of significant change in mean answer

1.2

133

Extent of fellows’ organizations’ use of research evidence

1 0.8 0.6 0.4 0.2 0 pe Executive Group

Figure 9.3 Change in extent of fellows’ organizations’ use of research evidence, physician executive stream, cohorts 1 to 4

Knowledge of research-based evidence

Amount of significant change in mean answer

0.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0

Figure 9.4 Change in extent of fellows’ organizations’ use of research evidence, by nursing and organization executives’ professional streams, cohorts 1 to 4

134

The Curriculum Elements

Fellows’ skill-set for assessing quality of evidence

Amount of significant change in mean answer

1.5

1

0.5

0

ne

oe Executive Groups

pe

Figure 9.5 Change in fellows’ skill-set for assessing quality of evidence, by professional stream, cohorts 1 to 4

Amount of significant change in mean answer

2

Fellows’ ability to promote use of research evidence in their organization

1.5 1 0.5 0

ne

oe Executive Groups

Figure 9.6 Change in fellows’ ability to promote use of research evidence in their organization, by professional stream, cohorts 1 to 4

The data from the tables and graphs support the notion that the program has indeed led the fellows to acquire new skill sets and knowledge. Although assessments for each of the modules have varied, consistently high values have been ascribed to Module 4 – Using Research-Based Evidence to Create and Manage Change. Continuing concerns had been expressed by fellows regarding the final module; specifically with regard to the Community of Practice material, a point that also was continually stressed in the focus groups at the final residency sessions each year. The program has responded to the

Evaluating the First Years

135

Knowledge of change management Amount of significant change in mean answer

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

ne

oe Executive Groups

Figure 9.7 Change in fellows’ knowledge of change management, by professional stream, cohorts 1 to 4

views expressed by the fellows, and there is much less emphasis now on Communities of Practice material. The survey data clearly identify a significant change in the fellows’ self-rating of knowledge and skill sets by cohort and by professional streams, especially with regard to knowledge of research evidence, skill sets for assessing the quality of research evidence, knowledge of change management, and the ability to promote the use of research evidence.

Do Fellows Improve Their Capacity to Collaborate as Evidence-informed Decision Makers between Professional Streams? In the current health care context it is becoming increasingly important and well recognised that greater levels of collaboration enhance the provision and quality of care. One of the key outcomes expected of the program, therefore, is to see if fellows have improved their capacity to collaborate as evidence-based decision makers. Tables 9.7 and 9.8 provide data to indicate the extent to which this is achieved. When the paper-based survey data is broken into the professional streams, statistically significant changes occur for nurses (n=34) and organizational executives (n=39), as shown in figure 9.8 (note that there were too few responses for physicians to determine statistical significance, although the data still showed the same direction).

136

The Curriculum Elements

Table 9.7 Capacity to Collaborate as Evidence-Based Decision Makers, Cohorts 1 to 4 Strongly Agree / Agree (rated on 5 point scale at the end of 4th residency, percentage) Results Assessed by Fellows

Cohort 1 Cohort 2 Cohort 3 Cohort 4

Stronger collaborative networks were built between professional streams in the program

90

86

80

86

Intervention Project created opportunities for interdisciplinary work and interprofessional collaboration

95

86

79

90

Source: Evaluation survey data, various years.

Table 9.8 Overall Assessment of Collaboration, Cohorts 1 to 4 Percentage Change over Time (baseline to 4th residency) Results Assessed by Fellows

Cohort 1 Cohort 2 Cohort 3 Cohort 4

Use of research evidence when collaborating with professionals in host organization (all the time, most of the time)

38+

60+

28+

17+

Use of research evidence when collaborating with professionals in other organizations (all the time, most of the time)

29+

44+

21+

17+

Source: Evaluation survey data, various years.

In all respects the survey data in the tables and in the graph reflect increasing levels of collaborative activity by the fellows in each cohort grouping and in the professional streams for the first four cohorts. Extensive collaborations are also evident in the work fellows have engaged in as part of the intervention projects.

The Intervention Projects Two members of the Evaluation Team undertook a systematic review of all the intervention projects completed by the first three cohorts (n=76). In addition to identifying a diverse range of initiatives among the host organizations, the researchers sought to quantify, to

Evaluating the First Years

Amount of significant change in mean answer

1.4

137

Extent of fellows’ use of research evidence when collaborating with professionals in other organizations

1.2 1 0.8 0.6 0.4 0.2 0

oe

ne Executive Groups

Figure 9.8 Change in extent of fellows’ use of research evidence when collaborating with professionals in other organizations, by professional stream, cohorts 1 to 4

the extent possible, a range of characteristics among the projects that could be associated with the overall expected outcomes of the EXTRA/FORCES program.2 Our initial work on conducting a systematic review of the intervention projects led to a conceptualization of an intervention project continuum that distinguished between (1) projects designed to develop an understanding of the health care environment and/or issues facing a host organization, (2) projects formative in nature that would, or could, be used as a springboard for a direct intervention, and (3) projects that were a direct intervention involving patients/ clients. Based on this continuum, the data in tables 9.9 and 9.10 emerged from the systematic review. One of the important points to take from these tables is that the intervention projects do not necessarily lead to outcomes being achieved within the period of the two-year fellowship. More longitudinal work is required to fully discern the effects of the IP on the organizations and the health systems of which they are a part. This is important when considering the impact of the EXTRA program on the health care system overall. In many cases our review was not able to easily identify the extent to which direct impact was achieved or not. We will need to go back to the respective fellows themselves 2 For the seventy-six projects, one final report was unavailable, one was unreadable, and a third had significant edits making it difficult to comprehend.

138

The Curriculum Elements

Table 9.9 The Intervention Project Continuum, Cohorts 1 to 3 Conceptual Development (1)

Formative (2)

Direct Intervention (3)

Total

Cohort 1

7

9

7

23

Cohort 2

8

11

6

25

Cohort 3

12

10

4

26

Total

27

30

17

74

Source: Evaluation data, review of Intervention Projects.

Table 9.10 Scope of Direct Impact of Intervention Projects, Cohorts 1 to 3 Specific Disease/ Illness Group

Hospital

Cohort 1

7

2

4

0

6

5

Cohort 2

8

2

2

2

3

7

Cohort 3

6

0

5

0

3

12

21

4

11

2

12

24

Total

Region

Province

All

Unable to Determine

Source: Evaluation data, Review of Intervention Projects. Notes: The term “scope of direct impact” refers to the impact on clients, patients, and residents of long-term care facilities. A considerable number of projects did not involve them directly but had a potential impact. There is notable increase in these projects in Cohort 3 Intervention Projects.

in order to more fully identify the extent of changes occurring as a result of the IPs. A large number of different groups participated in the intervention projects. Of the seventy-three projects that discussed the different disciplines that were involved, only ten projects stated that just one discipline was involved. Furthermore, as table 9.11 shows, a large number of different stakeholders3 were involved in the IPs. In all respects, the fellows appear to have improved their capacity to collaborate within their own organizations, with professionals in 3 Stakeholders were divided into internal and external groups or individuals that had a vested interest in the project. Examples of external stakeholders are unions, because of the impact on their members, the CIHI, which is interested in the data generated and reported, and professional colleges or associations. Internal stakeholders range from individual units or programs within an organization to specific groups of employees such as senior management, directors, managers, and doctors.

Evaluating the First Years

139

Table 9.11 Involvement of Stakeholders in the Intervention Project, Cohorts 1 to 3 Number of Stakeholders Involved

Cohort 1

n/a

0–5

6–10

11–20

20+

Total

0

14

7

1

1

23

Cohort 2

0

10

8

3

3

24

Cohort 3

1

14

9

1

1

26

Total

1

38

16

5

4

73

Source: Evaluation data, Review of Intervention Projects.

other organizations, among their peers (the other fellows), and through the development of the respective intervention projects. This collaboration is further supported by focus group discussions held every year in which the fellows, at four separate group discussions, have consistently stressed that one of the significant and, indeed, unexpected outcomes of their fellowship has been the extensive networking and the development of formal and informal relationships among the fellows themselves. Strong informal bonding and ties were made over the duration of the two-year fellowship.

Appling the Skills Learned and Their Diffusion beyond Formally Enrolled Fellows The fellows’ knowledge has been enhanced and their skill-sets improved. Has this then led to organizational change? As the research literature firmly attests, the extent to which change does occur will be highly contingent on a number of inter-related factors connected to context, content, and process. Numerous challenges may need to be overcome, and depending on the fellows themselves, a number of strategies can be deployed in an attempt to ensure that research evidence better informs decision making. Tables 9.12 and 9.13 identify data collected that address these results.

Challenges to Improving the Use of Research Evidence In the first round of surveys with cohort 1 we asked open-ended questions regarding the challenges being faced to improve the use of research evidence in the respective host organizations. Based on the

140

The Curriculum Elements

Table 9.12 Fellows Use of Research Evidence, Cohorts 1 to 4 Percentage Change in Use over time (baseline to 4th residency) Cohort 1 Cohort 2 Cohort 3 Cohort 4 Fellows who state they use research evidence (most or all of the time)

29+

76+

23+

17+

Fellows who state they use research evidence when collaborating with professionals in host organization (all the time, most of the time)

38+

60 +

28+

17+

Fellows who state they use research evidence when collaborating with professionals in other organizations (all the time, most of the time)

29+

44+

21+

17+

Source: Evaluation survey data, various years.

Table 9.13 Fellows Overall Assessment of Spread, beyond the Fellows, Cohorts 1 to 4 Yes versus No (rated at the end of the 4th residency, percentages) Cohort 1 Cohort 2 Cohort 3 Cohort 4 A change in the organizational culture, with more use of evidencebased decision-making

na

76

79

100

Strategies developed for colleagues to use and apply evidence

na

85

75

90

Host organization engaged in change management strategies based on the Intervention Project

85

81

71

95

Findings from the Intervention Project transferred beyond the host organization

50

76

50

48

Improved diffusion and use of evidence-based tools

na

95

83

79

Increased awareness of evidencebased decision making and its value in the host organization

na

95

100

100

Fellows who state their organization uses research evidence frequently, most, or all of time (between baseline and 4th residency)

4-

43+

20+

47+

Source: Evaluation survey data, various years.

Evaluating the First Years

141

responses, we then converted the open-ended question into closed items with five-point Likert scales for subsequent surveys of the following cohorts of fellows. Cohorts 2 to 4 were asked to rate the extent to which the challenges listed were indeed challenges they were facing in their own organization. Consistently over the next three years, clusters of challenges were identified by the fellows in cohorts 2, 3, and 4 (see column 1 in table 9.14).4 These findings are consistent with our observations from reviewing the intervention projects and with our discussions with the fellows from interviews and focus groups (note: we have listed briefly in the third column what we feel could be appropriate solutions to address the challenges).

Strategies for Transferring EXTRA Knowledge As with the question of improving the use of research evidence, we used the same approach to further understand the strategies fellows used to transfer EXTRA knowledge to the host organizations. In our first round of surveys with cohort 1 we had asked open-ended questions regarding the strategies used by the fellows to improve the use of research evidence in the respective host organizations. Based on these responses we then converted the open-ended question into closed items with five-point Likert scales for subsequent surveys of the following cohorts of fellows. Cohorts 2 to 4 were asked to rate the extent to which the strategies listed were strategies they used in their own organization. Numerous strategies were identified by the fellows, but over the next three years the strategies listed in table 9.15 were consistently stated as the most prevalent by the fellows in cohorts 2, 3, and 4.5 Again, it should be noted that these factors are congruent with our observations from reviewing the intervention projects and the information from interviews and focus groups conducted with the EXTRA fellows.

4 Note: there was a range of challenges identified by the fellows. These were clustered into the groupings shown in the table by members of the evaluation team. 5 Note: there was a range of strategies identified by the fellows. These were clustered into the groupings shown in the table by members of the evaluation team.

142

The Curriculum Elements

Table 9.14 Challenges to Improving the Use of Research Evidence in Host Organizations Identification of Issue

Explanation

Time pressures of the fellows

Constraints negating the fellows’ ability to do more knowledge transfer in the organization, despite their interest in doing so

No time for clinicians to stay on top of research

Time pressures of organization’s professionals limiting capacity to use research evidence, even though there may be an interest in doing so

Staff saying they don’t have time to use research

Constraints negating ability of staff to use research; everyone saying they are too busy

Time to develop a structure and support function

There may be interest and receptivity, but no formal supports have been developed, and no time has been made to develop them, to assist in a broader transformation.

Difficulty accessing research evidence in a timely way

Limited skill sets of staff to access research evidence

Staff willingness (and readiness) to use research

Reluctance (lack of receptivity) to use research Need to ask why this is the case and to introduce strategies to address this

Competing organizational priorities

While the use of research may be considered by everyone to be important, when up against competing priorities it does not get the time and attention necessary.

Lack of skills of staff

Limited skill sets of staff for using research evidence

Source: Evaluation survey data, various years.

Discussion The survey data for all the cohorts indicate that the fellows have increased their use of research evidence over the period of the twoyear fellowship. Some cohorts have shown a greater increase than others. Similarly, the fellows have identified that their own organizations have significantly increased their use of research evidence in different ways, including • • •

effecting a change in the organizational culture, developing strategies for colleagues to use and apply evidence, engaging in change management strategies based on the Intervention Project,

Evaluating the First Years

143

Table 9.15 Strategies Used to Transfer EXTRA Knowledge in the Host Organizations Informal tacit knowledge exchange Fellows leading by example with the use of research evidence in their own role and function Increasing awareness by talking about “evidence” at all levels of meetings Encouraging others to use evidence when presenting arguments Encouraging trainees to use evidence Asking questions, encouraging exploration of evidence to support ideas Informally transferring knowledge among colleagues Formal knowledge exchange Modifying content of presentations and discussions in formal meetings with senior management Modifying content of presentations and discussions in formal meetings with staff Use in development and use of Best Practice Guidelines Incorporating into procedures for patient care Source: Evaluation survey data, various years.

• •

improving the diffusion and use of evidence-based tools, and increasing the awareness of evidence-based decision making and its value in the host organizations.

As the tables have shown, the fellows have identified a number of challenges to effecting change in their organizations and a range of strategies they have used to enhance the use of research evidence in their organizations. In essence, this critical interface – mitigating challenges to effect change – is the hub of the program. Fellows are immensely appreciative of the program (survey, focus group, and interview data) and have learned new knowledge and developed new skill sets. The program theory is that this will lead to change in their organizations. While this certainly appears to be taking place, the precise contours of these changes are highly contingent on the organizational context in which each fellow functions. Interestingly, similar strategies are being deployed by the fellows to effect change, much of which is more focused on “informal” tacit knowledge exchange. There are no doubt considerable lessons to be learned about how these changes take shape and under what

144

The Curriculum Elements

organizational conditions, a key point that we will be addressing in the ongoing evaluation research. Data compiled from the review of EXTRA intervention projects in the first three years show the range of work being conducted by the fellows. Most of it is classified either as conceptual or formative development; much less of it is considered as direct intervention. What this highlights, however, is that the IP projects simply take time and that the full outcomes of the interventions will not be known until after the two-year fellowship period. Concomitant with this is the likelihood that (a) the outcomes expressed at the beginning of the project are likely to evolve over time, (b) many subtle and not so subtle changes may occur simply through the process of developing the intervention projects, and (c) the limited extent of internal evaluation of the IPs by the organizations may not fully uncover the depth and breadth of change occurring. There is much to learn from the details and nuances of the fellows’ experiences as they try to effect changes over time, much of which is not detailed in their IP documents.

Enablers: Mentoring and Desktop Support The mentoring activities and the desktop provide support for the fellows during the program. Importantly, they serve to support activities that contribute to the achievement of the overall program outcomes. Although many contingent and contextual factors are associated with these enablers, they have been consistently provided to the fellows to support their work. They are, in fact, integral components of the EXTRA program. Tables 9.16, 9.17 and 9.18 present survey findings that asked fellows at various points in their fellowship about the extent of their satisfaction with the mentoring functions and the desktop. More detailed information on the mentoring function is provided in Sam Shep,s chapter 8. Although some concerns had been expressed regarding the regional mentoring centres (RMCs) in the first year, there was a noticeable improvement in the assessment of the RMCs with future cohorts. Some new concerns have been observed, however, with cohort 4. The data shown in table 9.16 are consistent with discussions the Evaluation Team has had with fellows in the focus groups each year. Although it is always challenging to create an effective relationship that accommodates the individuality and expectations of both the

Evaluating the First Years

145

mentor and mentee, across all the cohort years there have generally been very positive relationships between the fellows and the mentors. The desktop has consistently been regarded as a valuable resource for the fellows, as evidenced by the data in table 9.16. Again, this is supported by other data from focus groups and interviews, with the additional acknowledgement that its role is especially critical immediately before and during the residency sessions.

The Overall Experience of Fellows with the EXTRA Program The evaluation team has conducted focus groups with each cohort at the end of that cohort’s second year (at the fourth residency). Four focus groups are conducted by two evaluation team members (two in French and two in English). Fellows are asked to comment on their experiences with the program overall on a number components – residency sessions, mentoring, intervention projects, inter-residency learning, networking/collaboration, information technology, and unexpected outcomes. Overwhelmingly, the fellows have had significant positive experiences from being involved in the program. Although there have been a range of challenges – the biggest being the time available to commit to the EXTRA work, especially the intervention projects – the fellows have been uniform in their appreciation for the program, and the EXTRA program staff especially, and the receptiveness of the program to addressing issues that have arisen along the way. If there is one area that still needs further refinement, it is that of mentoring; fellows desire a more consistent approach to mentoring across the country, since it appears that the regional mentoring centres do things differently. Complicating the picture is the fact that mentoring relationships themselves are very different from one another, depending on the needs of the mentee and the commitment of the mentor, among other things. Flexibility then, is both inherent and a requirement for a successful overall mentoring program. Overall, however, the fellows show a general high level of satisfaction towards the mentoring experience. Importantly, the dissatisfaction that was expressed in the first year of the program was addressed immediately by it, a responsiveness that continues in other areas such as refinements in the residency modules and the continual interest in having more engagement of the CEO of the respective host organizations.

146

The Curriculum Elements

Table 9.16 Mentoring and Desktop Support, Cohorts 1 to 4 Percentage Positive Assessment (Strongly Agree/Agree) On a Five-Point Likert Scale (rated at the end of the 4th residency) Responses

Cohort 1 Cohort 2 Cohort 3 Cohort 4

Fellows felt they were provided with effective and high quality self-directed learning and mentoring support Fellows felt they were provided with effective high quality desktop support

60

91

88

76

100

91

87

81

Source: Evaluation survey data, various years.

Table 9.17 Enabler: Regional Mentoring Centre Support, Cohorts 1 to 4 Support from Regional Mentoring Centre

Cohort 1, year 1 Cohort 1, year 2 Cohort 2, year 1 Cohort 2, year 2 Cohort 3, year 1 Cohort 3, year 2 Cohort 4, year 1 Cohort 4, year 2

Poor/Insufficient

Sufficient

Excellent

No response

7 8 2 2 0 0 0 3

24 13 29 29 25 24 17 15

0 2 7 10 16 14 24 6

1 2 0 2 2 2 0 0

Table 9.18 Enabler: Mentoring Support and Communication, Cohorts 1 to 4 Support and Advice from Mentor re ip

Communication with Mentor

Poor/ Poor/ Insufficient Sufficient Excellent Insufficient Sufficient Excellent Cohort 1, year 1 Cohort 1, year 2 Cohort 2, year 1 Cohort 2, year 2 Cohort 3, year 1 Cohort 3, year 2 Cohort 4, year 1

1 2 0 2 2 2 0

15 12 17 18 18 15 18

17 9 21 22 22 22 21

2 4 0 1 6 4 1

15 7 11 19 21 14 18

16 12 27 22 15 19 20

Cohort 4, year 2

7

5

13

4

6

15

Source: Survey data from the Regional Mentoring Centres.

Evaluating the First Years

147

Interviews conducted with the fellows over the years have further emphasized the commitment of the program to continuous improvement and the high level of appreciation for the program and the support from the EXTRA staff and for how the program has assisted the fellows in their own career paths and improved the extent to which research evidence is used in decision making. What has emerged consistently over the years has been the huge appreciation for the informal networks of relationships created and sustained by the fellows. Repeatedly we have been told in interviews and focus groups that the most significant and, indeed, unexpected outcome has been the extent and strength of relationships established across the country by the fellows. The fellows can now tap into the experiences of other fellows at any time during and subsequent to the fellowship and have a direct lens into the changes occurring in different health care systems. Relationships with faculty members, meanwhile, have continued and the connections made during the fellowship period itself with faculty was, and is, considered extremely valuable. In short, the fellows have been provided with an enormous resource base and networks of relationships that they would not have had otherwise.

The Next Five Years As previously stated, the predominant focus of the first few years of evaluative work has been on the fellows themselves; the extent to which they have acquired knowledge over the duration of their fellowship period. The evaluative data show that the first five years of the EXTRA program have seen the successful transfer of knowledge on the use and uptake of research evidence for decision making to the health executives. These fellows have taken the knowledge gained from residency sessions, mentoring relationships, networking, intervention project development, and the use of the desktop resources to effect changes in their respective organizations. There will be a greater depth of evaluative enquiry in the next five years, which requires targeted evaluation activities in addition to the current data being compiled. The various strands of evaluative evidence and anecdotal accounts suggest that, owing to a variety of individual, context, content, and process factors, some fellows have had more success than others in bringing about change in (a) their own day-to-day activities, (b) their sphere of influence, (c) their organization, and (d) the broader health care system in which they function.

148

The Curriculum Elements

There is both a desire and a need to more fully delineate the nature and extent of the changes occurring in order to evaluate the degree of success of the EXTRA program. Importantly, there is a need to apply the evaluative lens to understanding the changes that have occurred beyond the fellows own knowledge, attitudes, and behaviors. Thus, four key questions will be examined in the unfolding evaluative research: 1 What changes have occurred as a result of using research evidence for informing decision making by others in and outside the host organization? 2 What changes have occurred in the use of research evidence for informing decision making in the broader health care system (i.e., in other organizations and the system overall?) 3 How have the changes in (2) and (3) occurred? (what works and what doesn’t and why). 4 To what extent have the program enablers – the mentoring component, the Desktop, and the Intervention Projects – contributed to these changes? These questions frame a number of specific evaluative projects that are being conducted over the following year. These include, for example, examining the role of mentoring, the changes occurring through the intervention projects, the role of the desktop, the uptake of EXTRA and its components, leveraging by fellows (including career paths), the utility of the program from the CEO perspective and the detailed nature and extent of knowledge transfer by the fellows.

Summary The EXTRA/FORCES program has brought about change in a diverse range of health care organizations across the country. This change began with knowledge about research evidence and its use being transferred to fellows and the fellows themselves then applying this knowledge to effect change in their organizations. The change has been brought about through explicit formalized activities and more informal approaches that have sought to promote and enhance the use of research evidence by colleagues in different settings – such as in group discussions, report writing, access to research articles, and

Evaluating the First Years

149

so on. Although both approaches contribute to changing organizational culture, the pace of change will vary according to the context of the respective organizations. Put simply, some organizations will be more receptive to change than others. Organizational receptivity, in fact, is at the heart of the EXTRA program. There is a need and desire to learn more from the experiences of the fellows so that (a) the program itself learns what works to effectively increase the use of research evidence (and what does not work well), and (b) other health care organizations not involved in the EXTRA program can learn from the experiences and effect change in their own contexts. Such changes can take time. From an evaluative perspective, it is more likely that the full extent of changes occurring as a result of health professionals being in the program will not be fully realised until after their two-year fellowship. The first few years of the evaluation have documented positive changes occurring for the expected outcomes as articulated at the inception of the program. We are now at a point where we will focus much more intently on the depth of changes – the how and the why of effecting change – and the longerterm effect of the fellowship on the host organizations, the fellows themselves, and the broader health system in which they function.

Appendix A Evaluation Team members (2005–8) • Malcolm Anderson (Queen’s University) – Lead Evaluator • Lynda Atack (Centennial College) • Dorothy Forbes (University of Alberta) • Susan Donaldson (Susan Donaldson and Associates) • Melanie Lavoie-Tremblay (McGill University) • Manon Lemonde (University of Ontario Institute of Technology) • Lorna Romilly (Lorna Romilly and Associates) • Lyn Shulha (Queen’s University) • Ingrid Sketris (Dalhousie University) • Richard Thornley (Alberta Heritage Foundation) • Stephen Tomblin (Memorial University) Ingrid Sketris holds a Chair in Health Services Research funded by the Canadian Health Services Research Foundation and the Canadian

150

The Curriculum Elements

Institute of Health Research, co-sponsored by the Nova Scotia Health Research Foundation. We would like to thank all mentors and fellows who participated in the evaluation study.

References Anderson, M. 2006. The evolution of innovations in health care Organizations: Empirical evidence from innovations funded by the Change Foundation. Research manuscript prepared for The Change Foundation. Toronto. Anderson, M., L. Atack, S. Donaldson, D. Forbes, M. Lavoie-Tremblay, M. Lemonde, L. Romilly, L. Shulha, I. Sketris, R. Thornley, and S. Tomblin. 2004. The extra /forces program evaluation design. Working document. Kingston, on: Queen’s University. Canadian Academy of Health Sciences. 2009. Making an impact: A preferred framework and indicators to measure returns on investment in health research. Report of the Panel on the Return on Investments in Health Research. Design Working Group. 2003. Final report of the extra /forces Design working group. Canadian Health Services Research Foundation. Donaldson S.I. 2003. Theory-driven program evaluation in the new millennium. In S.I. Donaldson and M. Scriven, eds., Evaluating social programs and problems: Visions for the new millennium, 109–41. Kirkpatrick, D.L. 1994. Evaluating training programs: The four levels. San Francisco, CA: Berrett-Koehler. – 1998. Another look at evaluating training programs. Alexandria, va. American Society for Training and Development. Patton M. 1997. Utilization-focused evaluation. Thousand Oaks, ca: Sage Publications Pettigrew, A.M., E. Ferlie, and L. McKee. 1992. Shaping strategic change: Making change in large organizations; the case of the National Health Service. Thousand Oaks, CA: Sage Publications. Stake, R.E. 2004. Standards-based & responsive evaluation. Thousand Oaks, CA: Sage Publications.

section three

What Regional Leaders and Fellows Say about the Experience Chapters 10, 11, and 12 present perspectives on the EXTRA program from highly engaged health care executive leaders. They recount the stories from three regional health authorities in Canada that have actively promoted the program and enrolled significant numbers of senior employees as fellows. In each case, the authors look back at the pay-off of the program from an organizational point of view. In chapter 10 the CEO of the Winnipeg Regional Health Authority, Brian Postl, looks in detail at how three intervention projects had an impact on change within health services delivery in the region. He analyses how the teachings of the program, together with the development of intervention projects, serve as a valuable resource in bringing about changes in health services delivery. In chapter 11, Denis Roy and his colleagues from Montérégie, one of the largest regional health networks in Quebec, assemble the experiences of the Montérégie fellows on the program. They describe how the regional health authority used the EXTRA program as a lever to develop leadership capacity within the region. Chapter 12, by Chris Power and colleagues from the Halifax Capital Health region, describe how the efforts and interventions of the fellows came together with the overall “strategy quest” undertaken in the region. This chapter explains that in a learning organization the presence of fellows with visible change projects aligned to the strategy can help to build common commitment to better service, better integration, and better performance in a region with significant health status challenges.

152

What Leaders and Fellows Say

Chapter 13 presents viewpoints from three EXTRA fellows who share the challenges of transferring learning to a real organizational context. The authors provide valuable insights to connect more strongly the EXTRA learning experience with the organizational context to increase program impact. They also suggest different ways by which the program might more completely engage their host organizations. In addition, they offer thoughtful contributions on how fellows may further develop their skills for evidence-informed management beyond the timeline of the program. The volume concludes with individual and collective reflections and aspirations for the program from three fellows who span the country geographically and span clinical, administrative and policy perspectives. Beginning in the West, we hear from Ward Flemons, a 2008 EXTRA graduate and respirologist at the Calgary-based Foothills Medical. Moving east, we hear from 2007 EXTRA graduate Andrea Seymour, assistant deputy minister, supply and services, government of New Brunswick and former vice-president of health information and chief information officer with the River Valley Health authority in Fredericton, New Brunswick. Finally, from central Canada we hear from physician executive and 2007 EXTRA graduate Carl Taillon, who is director-general of medical and university affairs at the Centre Hospitalier Univérsitaire du Québec in Quebec City.

10 Building Capacity for Change Examples from the Winnipeg Regional Health Authority brian postl

The management and delivery of health care services and programs is becoming increasingly complex. For those of us operating within a regionalized delivery model, this complexity is accentuated as we deal with multiple sites, programs, and levels of accountability. We are accountable to our constituents – those to whom we provide services – and we are accountable to government, from whom most of our funding originates. To successfully execute these accountabilities, we need to acknowledge the value of using evidence to guide decision making and to recognize that evidence comes in many forms and currencies. However, it is one thing to recognize the theoretical value of evidence and information and another to apply it in a practical and useful manner in the planning and decision-making processes. In some cases, there is a misunderstanding of what constitutes evidence, and there may be some reluctance to use, or a mistrust of, the evidence that is available. Additionally, acquiring and using evidence to support a decision or policy may be seen as just one more step that reduces the time available for other key activities, instead of an investment to improve the quality of the decision. The board of the Winnipeg Regional Health Authority (WRHA) has recognized the need to incorporate evidence into decisionmaking processes and is committed to exploring opportunities that would increase its use, particularly at the management level. It appeared to the WRHA that the EXTRA program would be an ideal

154

What Leaders and Fellows Say

vehicle for achieving the goal of increasing the use of evidence within the organization. It was further expected that the region’s knowledge transfer process would benefit as a consequence. Since 2004, the year the EXTRA program was introduced, the WRHA has sponsored the participation of six members of its management team in the program; in fact, the WRHA has enrolled at least one of its staff in the EXTRA program in each of the years from 2004 to 2008. When selecting potential fellows, the senior management team considered the extent to which candidates could function as change agents within the organization. Each needed to be a decision maker in his or her own right. He or she was also expected to influence the decisions of others – most notably senior management, but direct service providers too. The intervention projects that were chosen by the participants were expected to reflect the WRHA’s strategic directions and to align with regional organizational priorities and provincial priorities. Each of those six fellows has made significant and ongoing contributions to the way information and evidence is used at the management level within the region. Two fellows and their work have been selected for further discussion here. They are • •

Dr Brock Wright (2004): Striving for Excellence in Patient Care Ms Trish Bergal (2006): An Evidence-Based Approach to Improving Hospital Patient Flow

When Dr Wright submitted his application to the EXTRA program in 2004, the WRHA had just begun to form its patient safety team and was developing its regional integrated patient safety strategy. Patient safety was identified as a focus within the region’s strategic plan, with the following specific goal: “We will evolve a culture and system that focuses on learning and collaborative improvement where patient safety is the primary focus for all staff.” Subsequently, patient safety was declared an organizational priority and appeared on the provincial radar screen as well. This is not surprising: by then, patient safety had become a major issue at national and international levels, and the estimated volume of adverse events was shocking and sobering. In 2004 Baker, Norton, et al. found that 7.5 percent of adult patients admitted to hospital experienced an adverse event and that 37 percent of these were preventable. For the WRHA, there was a real urgency associated with improving patient safety; in 1994, twelve children

Building Capacity for Change

155

had died at the Winnipeg Health Sciences Centre’s pediatric cardiac surgery program. When this tragedy occurred, the WRHA did not yet exist as an entity. However, in 2004 we knew that we as an organization had to take steps to ensure that an event such as this could not happen again. At that time, Dr Brock Wright was chief operating officer of the Health Sciences Centre. His interest in patient safety generally, and in particular within the centre, was timely and especially relevant. Of course, in addition to the individual human cost associated with adverse events, there are the business costs (whether to the organization itself, to the system, or to society as a whole) that are incurred as a result of medical error. Costs can relate to readmissions, additional physician visits, increased absenteeism, loss of income, and so on. These costs are compounded by the reputational losses that affect morale – not only within the individual site where the event has occurred but also across the entire organization. While improving patient safety at the Health Sciences Centre was fundamental to Dr Wright’s EXTRA intervention project, the project was intended to improve other aspects of organizational performance as well, including staff safety, productivity, and patient and staff satisfaction. It was designed to affect several different structures and processes, thereby achieving specific performance targets. While Dr Wright’s intervention was targeted to a specific site (Health Sciences Centre), the expectation was that the “lessons learned” would be generalized to all sites and to the region as a whole. Dr Wright developed a multi-faceted intervention model that acknowledged two key facts: 1 Achieving the stated objectives would depend on a multitude of factors that would each attain individual success. However, and perhaps more importantly, each of those factors would interact with others to maximize success. The whole truly is greater than the sum of the parts. 2 All the factors would have affect the culture of the organization, and the culture of the organization would influence the extent to which changes are accepted. Dr Wright’s model identified nine components: 1 Structure and decision-making processes 2 Patient safety projects: small wins

156

3 4 5 6 7 8 9

What Leaders and Fellows Say

Knowledge translation Performance management Infrastructure Communication Program consolidation Reporting and review Strategy development and implementation

Actions were developed for each of the nine components, with the expectation that collectively they would lead to the overall success of the intervention. In 2006, when Ms Bergal submitted her application to the EXTRA program, the WRHA was continuing its struggle with a problem that would not go away; namely, length of stay in the region’s acute care hospitals. The WRHA was not unique in this: virtually all regions and sites must contend with the length-of-stay battle. When we compared our performance with that of our peers in reports such as the CIHI/HayGroup Benchmarking Report, we saw the WRHA reflected in a less-than-favourable light. Our patients were staying in our hospitals longer than patients in hospitals in other jurisdictions. This was somewhat confounding to us because, historically, the WRHA has had a generous supply of institutional beds (both acute and personal care) and a comprehensive array of community-based services, including home care. We explored various reasons that we thought might explain our apparently inflated demand for beds. We looked at population-based factors, such as the age and sex of our patients and population, socio-economic factors that could have an impact on demand, and the relative size of at-risk populations. We also looked at how efficiently our sites were functioning relative to admission/discharge, referrals, timeliness of diagnostic interventions, and other factors. The bottom line: we could explain some of our increased utilization by some of the variables but remained unsure of how best to address the more complex and inter-related issues. Ms Bergal acknowledged that improvement in hospital utilization (length of stay) is continually sought. She noted that the timely and efficient movement of patients into, through, and out of the acute system – a concept currently described as “flow” – was necessary to balance demand with capacity, which in turn resulted in a more efficient and effective use of scarce resources. In addition, minimizing

Building Capacity for Change

157

a patient’s stay in hospital is considered to be better for the patient, as well as for the system. The issue was acknowledged in the WRHA board’s strategic direction related to treatment and support, which included a specific goal related to length of stay: “We will have reduced length of stay to meet targeted benchmarks” (Winnipeg Regional Health Authority 2005). As with Dr Wright’s initiative, Ms Bergal’s was consistent with one of the region’s organizational priorities as well: access. It was also in line with one of Manitoba Health’s priorities, which mandated a 10 percent reduction in the region’s numbers for length of stay. Ms Bergal’s initiative focused on the implementation of the Utilization Management (UM) System, a computerized system introduced by Manitoba Health that was expected to capture quantifiable bed utilization information that would reduce length of stay and improve patient flow. The UM System was implemented on a phased basis across the WRHA from November 2004 to February 2008. Ultimately, it was used by all adult programs at all acute sites – 72 patient care units with a total of 1,786 beds – resulting in a regional approach to utilization management. Analysis of the data coming out of the UM System early in its implementation was disappointing; despite considerable resources allocated to its implementation, results (reduced length of stay) were not meeting expectations. Ms Bergal’s intervention project, which began mid-way through the UM System implementation project, was intended to examine why the UM System was not producing those anticipated results. In contrast to Dr Wright’s intervention, which was site-specific, Ms Bergal’s was designed for regional application from the outset. As noted earlier, her project objectives were consistent with WRHA strategic directions and organizational priorities, as well as with provincial priorities. Before designing and implementing their interventions, both Dr Wright and Ms Bergal undertook extensive qualitative research in the form of comprehensive literature reviews. In Dr Wright’s case, the review identified issues that would affect the volume of medical errors, the potential to reduce that volume, and actions that would produce the desired results. Through her literature search, Ms Bergal was able to address the concept and efficacy of a utilization management tool, as well to identify a number of the components that affect patient flow. These factors included “a tone of expectation from senior

158

What Leaders and Fellows Say

leadership, interdisciplinary collaboration, stream-lined discharge planning practices, utilization data analysis coupled with the establishment of utilization targets/goals” (Bergal 2008a). Both fellows found that organizational change and successful implementation of an evidence-informed initiative are significantly influenced by organizational culture and structure. In both cases, leadership was noted as key in managing change within a complex organization. In any organization, change is a difficult process. When the organization is large and complex, with many levels of stakeholders, change becomes particularly challenging to implement and stressful to experience. Having a CEO state that “we will do things differently from now on” is not likely to facilitate a successful change process. As well, many staff members perceive new change processes as the “flavour of the month” when compared to day-to-day work imperatives. In order to successfully manage change, the organization as a whole must accept the need for it and believe that it will improve outcomes. Essentially, a shift in values and norms in the members of an organization must precede successful process change. For example, for a shortened length of stay in hospital to occur, it must be first considered a good thing for the patient, as well as good business for the organization. In the course of their interventions, both Dr  Wright and Ms Bergal experienced first-hand the difficulty of implementing a fundamental shift in the “way we do business.” The evidence-based environment became an operational imperative for both. Evidence – information that supports the changes that are to be implemented – must be provided to all stakeholders. However, it is not sufficient to simply provide the evidence that the suggested change will have a positive outcome. The leadership in the organization must be seen to support the change that has been proposed. While the importance and validity of evidence-informed decision making may be acknowledged, the impetus to act on what is known is not always there. In other words, the organization must divest itself of the “Do as I say, not as I do” culture. Addressing and responding to the WRHA management team’s expectations of the UM System was perhaps one of the greatest challenges faced by Ms Bergal. The system was supposed to be the magic bullet: the information it provided was expected to result in reduced length of stay very quickly. However, no easy answer was provided by the UM System; in fact, it ended up posing additional questions that required further exploration. It illustrated that managing length

Building Capacity for Change

159

of stay and managing patient flow were complex, inter-connected processes “comprised of multiple inter-acting variables which have complicated relationships and inter-connectedness with each other” (Bergal 2008a). It also illustrated a fundamental difference between a traditional approach to managing length of stay, which focuses on predict and control methods, and flow, which looks at process. There was no easy fix: we had to take the information that the UM System provided and dig deeper to find answers. Additionally, the manner in which the UM System was implemented posed inherent difficulties. As noted earlier, the WRHA employs a matrix management model that includes multiple sites and multiple programs at each site. Rather than focusing on a single site, it was decided that implementation of the UM System would occur on a program-by-program basis across sites. Following this approach, we lost the opportunity to ensure that the concept and value of the UM System was embraced at one site by a large pool of workers – something that has been referred to as the “contagion of acceptance” (Bergal, personal communication, 2009). In fact, we diluted the level of acceptance by implementing the system over many sites at one time. Ultimately, three hypotheses about problems were put forward as potential explanations for the lack of results: 1 Expectations from site senior leadership were not understood by direct care staff. 2 The value of the UM System was not being experienced. 3 There was little change in work flow processes (the system was not integrated) (Bergal 2008b). In order to address these challenges, it was decided to immerse the UM implementation project team in the practice setting in order to encourage the integration of the UM System with other processes designed to improve patient flow. This was done on a site-by-site basis. Small-group education and learning were emphasized, which fostered a greater appreciation of the opportunities and information provided by the UM System. There was still no magic bullet, but the inter-relationships between the factors that drive length of stay and patient flow were being explored and understood. Direct care providers were asking, how can we change things to make them work better? In the end, all units at all acute sites participated in this

160

What Leaders and Fellows Say

embedded-learning approach to using the UM System, and all participants derived significant benefit. If this was a fairy tale, the story could end here, with clear evidence of reduced length of stay and improved patient flow. However, that has not yet happened. Nonetheless, important changes were observed at individual units within sites: •

•

•

The value of the information generated by the UM System was acknowledged, and it was used to initiate change. New work processes were created that would have a positive impact on length of stay and patient flow. Self-generated innovations began to occur within units, and this enhanced networking opportunities.

Was the UM System intervention project successful? The answer is, “It depends.” The WRHA continues to struggle with length of stay in our acute sites, so the magic bullet remains elusive! However, like patient flow itself, the intervention was a process, and a great deal was learned as we went through that process. We learned that even when implementing regional programs, it is sometimes best to start at a single site, so that there is a concentration or contagion of acceptance. We learned that leadership and stewardship are paramount in the success of any new initiative. Further, leadership must not be confined to the CEO or the COO; champions must be nurtured all along the continuum, including – perhaps most importantly – direct care providers. We learned that there is no magic bullet: complex issues, with complex interdependencies and interactions, require complex solutions. And we learned that outcome indicators like length of stay do not always adequately reflect the return on our investments. With respect to the UM System itself, “use of the system has helped the WRHA move from a culture of blame to a culture where health care providers feel empowered to address and manage length of stay issues” (Currie 2008, 14). It taught us to avoid looking for a simple solution and to look for the complexities that make life interesting! Patient safety was receiving a lot of attention in health care when Dr Wright developed his EXTRA intervention project, Striving for Excellence in Patient Care. However, addressing the complexity and multitude of variables that the issue enveloped was no easy task. “How would you go about trying to improve the performance of an organization facing the following challenges: an overburdened staff,

Building Capacity for Change

161

too many customers (so customer satisfaction is not a priority), overregulation requiring negotiation with multiple stakeholders in order to introduce change, rudimentary information systems as well as an aging infrastructure, a limited ability to raise capital, and finally, the pressure of intense media scrutiny that focuses only on the organization’s failures, not its successes” (Wright 2006, 1). This statement describes the environment Dr Wright faced while implementing his intervention project. In essence, he had to contend with the same difficulty as did Ms Bergal: a complex issue affected by various interdependent and interactive variables that could not be resolved with a simple solution. Once again no magic bullet, no quick fix. Dr Wright went on to identify five specific challenges: 1 2 3 4 5

Competing priorities; doing things better, faster, cheaper Professional and program/department autonomy Stakeholder interests and influence Defining and understanding error Avoid being perceived as “flavour of the month’ – developing change that is sustainable (Wright 2008)

Dr Wright determined that to be successful in redesigning processes, staff from all levels in the organization would need to be engaged and work together across programs. However, programs and departments within health care organizations tend to function as silos and are often resistant to working together. As well, certain professional groups are very hierarchical; managers are sometimes uncomfortable allowing their front-line staff to play a meaningful role in designing how work is done. Generally speaking, it can be challenging to change how health care is provided in our public system, since there are many stakeholders who claim the right to influence the decisions that are made. Governments, media, accrediting bodies, licensing bodies, unions, associations, and others all believe that they should play a role in shaping how the system operates. Providers who want to initiate or resist change may be able to seek and find support for their particular concerns based on the sheer numbers of stakeholders alone. Redesigning processes to do things better, faster, and cheaper can often be achieved, but sometimes priorities conflict. During the course of Dr Wright’s intervention project, another system priority emerged: patient access. There was a risk that the pressure to increase

162

What Leaders and Fellows Say

throughput in an effort to reduce wait-times could be viewed as a threat to patient safety. To address such concerns, efforts were made to position long wait-times as a legitimate patient safety issue. In order to improve performance through a focus on patient safety, Dr Wright developed a strategy, informed by evidence, that involved several different patient safety projects. In combination, these projects aimed to shift the culture of the organization to be more safetyand patient-focused. Each of the nine components of Dr  Wright’s intervention model was addressed. The intervention strategy was vetted and adopted by the leadership of the organization. Focus groups involving staff from all levels helped to shape the strategy, and in the process a “burning platform” for change was established. An appeal was made to the intrinsic desire of most health care providers: to want to make a difference. The organizational structure and key decision-making processes were changed to make the organization more responsive and better able to implement the strategy. Several human resources initiatives were launched in an effort to link performance expectations with the goals of the intervention project. Infrastructure changes were completed and programs were consolidated to contribute to enhancing patient safety. Investments were made in enhancing the knowledge translation capacity of the organization, and a comprehensive communication strategy was developed to support the project. Specific patient safety projects were initiated that would bring staff together from different programs and different professional groups. In some cases, students from the Faculty of Engineering at the University of Manitoba were involved in the projects to provide process re-engineering expertise. The culture change strategy adopted by Dr Wright was based on the concept of “small wins.” By achieving and celebrating small project successes, an interest in the larger goal of patient safety and system improvement could spread more effectively across the organization. The following are some examples of the specific patient safety projects/culture: •

•

Patient safety was established as a priority within the Health Sciences Centre and within the WRHA as a region, and a communication strategy was implemented that identified it as such. Structures and processes were modified to ensure the Health Sciences Centre was more responsive to patients and staff and was more decisive when issues were identified.

Building Capacity for Change •

•

•

•

•

163

An environment was created that ensured that the projects, and patient safety generally, were seen to be sustainable, as opposed to being a quick fix/flavour-of-the-month approach to problem-solving. Strategies to enhance knowledge translation and diffusion were developed and deployed. Specific projects intended to generate small wins were initiated to produce an attitude of success. The Health Sciences Centre worked in collaboration with the region to involve patients in patient safety strategies, strengthen critical clinical occurrence reporting, and improve information technology systems to improve patient safety. Quantitative measures to gauge success were developed and the information generated was shared.

Without question, this intervention, with its targeted activities, was extremely ambitious, even given its five-year mandate. The process began in the fall of 2004 and is in fact still going on. Was it able to meet all of its defined objectives? The answer is a qualified “Yes.” Progress has been made on all fronts, but much work remains to be done before it can be deemed completed. Work in the areas of patient safety, staff safety, staff productivity, and patient and staff satisfaction is ongoing. This brings us back to the idea of change and the notion that change is continuous. Within that context, work in these areas can never be completed. There are always improvements to be made and processes that can be streamlined, and everyone can be “happier.” And that is how it should be. Participation in the EXTRA program demanded a substantial investment on the part of Dr Wright and Ms Bergal, as well as from the WRHA as an organization. Whenever you invest, you must assess returns. Certainly, their involvement in the EXTRA program provided benefits not only to Dr Wright and Ms Bergal as individuals but to the organization as a whole as well. Both Dr Wright and Ms Bergal learned the value of using evidence to make the case for change. Evidence was also crucial in helping to identify and develop intervention strategies. In addition, the importance of ensuring that staff members always have a sense of ownership over any strategy that is proposed and the value of celebrating and recognizing success along the way were recognized. Finally, while it is important to be clear about the goals of an intervention, it

164

What Leaders and Fellows Say

is equally important to be flexible about the strategies employed to achieve those goals. Evidence must continue to inform the process if the project is to be successful. Through their experiences, it became clear that change is a process that must be nurtured and championed by those in a position to affect its acceptance. Change, no matter how large or how small, affects people, and change can be very frightening. Nevertheless, change is a constant: a truism certainly, but one that must be acknowledged. Consequently, to facilitate change within an organization as large as the WRHA, consideration must be given to adopting a model for change: a formalized process to initiate, implement, and manage change within an organization. Furthermore, change that is not sustainable – and not supported by organizational structure – is not practical and a waste of resources in a time when resources are becoming increasingly scarce. Consequently, in addition to a change model, a sustainability model must also be adopted to ensure that the changes proposed are consistent with the direction in which the organization is moving. Needless to say, evidence and information are the backbones of both structures. Without question, the WRHA’s participation in the EXTRA program over the last several years has had a profound impact on our use of evidence to guide our decision-making processes at both the strategic and the operational levels. Various initiatives have been introduced and implemented that facilitate the use of information by staff at all levels, from senior management to the front-line care provider. These collective changes all use evidence as their base and have facilitated the maturation of the WRHA into a learning organization, moving the region forward in a way consistent with evidence-based goals. Key among these initiatives are the following: •

•

•

The development of a project management office, facilitating the development of staff expertise in learning technology. The introduction of a regionalized integrated patient safety strategy with emphasis on culture change, direct patient involvement, learning from clinical practice and promoting change in care delivery. The creation of the Division of Research and Applied Learning, with units specializing in research and evaluation, health information services, quality and accreditation services, patient safety

Building Capacity for Change

• •

•

•

• •

165

and library services. As part of the its commitment to learning, the WRHA made a significant contribution ($500,000) to the University of Manitoba/Health Sciences Centre campus, Neil John MacLean Library Capital Fund. Full implementation of the Utilization Management System. Development of a wait list/wait time management process (in collaboration with Manitoba Health and Healthy Living), with a current focus on selected diagnostic and surgical interventions, but considering moving to referrals and other related areas. Enhancement of primary care and support for family practitioners, including the creation of a family physician action council, direct MRI referrals, and an integrated family medicine/primary care program. Enhancement of specialty care access, including an initiative to bridge generalist and specialist care, the development of a catalogue of specialist services, and our “booking ahead” initiative (the patient is given a date for services far into the future). Participation in the Royal Roads University program. Piloting of a new evidenced-based initiative prioritization protocol as part of our annual regional health plan process.

The next stage in activities to improve quality will be to ensure that the whole will be greater than the sum of its parts. To that end, the region has reorganized and recentred its focus on the concepts of innovation and integration. Indeed, the board is reorganizing, with one of the major committee functions being “Quality, Patient Safety and Innovation.” At the strategic level, it is understood that information and evidence are prerequisites for the development of a relevant strategic plan. Consequently, when the WRHA Board began the renewal of its strategic plan in 2009, it was provided with a comprehensive range of evidence and information to guide the development of the organization’s strategic directions, goals, and objectives. The findings from the 2009 Comprehensive Community Health Assessment were available to help shape board priorities. Any presentations made by senior management or by the various program teams were, and still are, expected to be supported by evidence. Decision makers, whether at the program level, the site level, management level, or the board level, are hungry for the information they need to make the best decisions possible. Similarly, front-line

166

What Leaders and Fellows Say

workers look for the information they need to provide the best care possible. All depend on evidence being provided to them. The Winnipeg Regional Health Authority is a relatively young organization, having been in existence for less than a decade. Within a short time, it has evolved from an organization that was guided more by intuition than evidence to one in which the evidence is becoming increasingly more important. Terms like “research,” “evidence-informed,” “best practice,” “evaluation,” and “benchmarks” have all found their way into our vocabulary and have come to reflect our culture.

References Baker, R.G., P.G. Norton, V. Flintoft, R. Blais, A. Brown, J. Cox, E. Etchells, W.A. Ghali, P. Hébert, S.R. Majumdar, M. O’Beirne, L. Palacios-Derflingher, R.J. Reid, S. Sheps, and R. Tamblyn. 2004. The Canadian Adverse Events Study: The incidence of adverse events among hospital patients in Canada. Canadian Medical Association Journal 170(11): 1678–86. Bergal, Trish. 2008a. An evidenced-based approach to improving hospital patient flow. EXTRA Intervention Project Report 2008: 1–3. – 2008b. An evidenced-based approach to improving hospital patient flow. Presentation at the 2008 EXTRA Program Intervention Project Preliminary Report session, 19 February 2008. Currie, Jan. 2008. Introductory comments on behalf of the Winnipeg Regional Health Authority. Presentation at the 2008 EXTRA Program Intervention Project Preliminary Report session, 19 February 2008. Wright, Dr Brock J. 2006. Achieving excellence through patient safety at Winnipeg’s Health Sciences Centre. EXTRA Intervention Project Report 2006: 1–51. – 2008. Achieving excellence through patient safety at Winnipeg’s Health Sciences Centre. Presentation at the 2008 EXTRA Program Intervention Project Preliminary Report session, 2006. Winnepeg Regional Health Authority Strategic Plan. http://www.wrha. mb.ca/about/files/StrategicPlan_Apr05.pdf.

11 The extra Experience in Montérégie denis a . roy, sylvie cantin , and stéphane rivard

Background To best appreciate the impact and implications of the EXTRA program at work in the Montérégie region, it is important to first understand the context in which the EXTRA fellows developed and applied their intervention projects. The task of setting up local health and social services networks in the Montérégie region – a project entrusted to the region’s Agence de la santé et des services sociaux in December 2004 – presented the agency’s directors with a monumental task. It also presented an opportunity to transform and modernize the organization.1 The project took place within the context of a triple-tiered governance model – provincial, regional and local – that is characteristic of Québec’s health and social services system. In order to successfully amalgamate and create local health and social services centres in such a large region, major efforts had to be made to engage “players”

1 The health and social services system in Montérégie, Québec, serves a population of 1.4 million. At the local level, there are eleven cssss (health and social services centres) with their own boards of directors; their mandate is to coordinate a local service network. Each csss is responsible for the population in a specific area, of a size varying from 25,000 to 225,000 people. At the regional level, there is a university-affiliated hospital, and there are 6 specialized rehabilitation centres and over 450 community organizations. The agency, which includes the regional public health team, governs the entire regional health and social services system.

168

What Leaders and Fellows Say

at the ground level in the local institutions across the region. To achieve political balance within the context of such a large change, the agency had to focus on helping local institutions develop autonomy. The complexity of the project was heightened by the need to deal with changes to union affiliations and to bargain for new collective agreements. Adding to the challenge were the many changes that all came into force at the same time: the introduction of management agreements, greater attention to risk management and to the handling of complaints, and certification procedures. All of this was driven by the need to balance budgets and to deal with the reality of shortages of medical staff and professional and technical personnel. Reconciling the expectations of local partners with those of government authorities often imposed difficult choices for senior managers and managers in the field. To support decision making, the Comité de coordination stratégique de la Montérégie (CCSM) (Montérégie Strategic Coordination Committee) was formed, bringing together all the general managers of the local and regional institutions, as well as the managers of the agency. Forming the committee was seen as the ideal way to engage in dialogue about strategic choices. The CCSM then formed collaboration networks structured on the major areas of intervention (clinical/administrative networks) and on management functions (resource networks). Managers drawn from all the institutions in the region and from the agency were brought together in these networks, where they collaborated within the framework of work plans that were focused on best practices in each business area. Following the abolition of the regional board and the creation of the Health and Social Services Agency, the nature of the contribution from the regional authority and, consequently, the agency’s structure and services supply were redefined to better support continuous performance improvement in the system. The creation of a network co-ordination authority and an information and knowledge management department, along with the redefinition of the public health services supply, were other significant developments. Even though the agency was faced with organizational, administrative, political, and media pressures, progress was achieved in terms of services to the population. In fact, emerging innovations in the field and various achievements by the institutions were recognized with awards at the provincial level. The agency’s leadership in evidence-informed

The Experience in Montérégie

169

decision making was also confirmed when it was presented with the Health Services Research Advancement Award in 2007. It is within this general framework of transformation and decentralization that we can fully understand and appreciate the repercussions of the EXTRA program in Montérégie. The EXTRA program launch coincided with the agency’s structural transformation. The desire to encourage local initiatives and evidence-informed decision making was entirely convergent with the objectives of the EXTRA program, which provided a not-to-be-missed opportunity to develop the ability to use research-generated data in Montérégie. It was actively promoted as a potential catalyst for change that would be in keeping the organization’s overall strategic direction. To this end, the agency focused on new organizational leaders who were firmly established in their respective environments and able to introduce the innovations that they considered the most appropriate in their specific contexts. In fact, the president and general manager of the agency was the very first decision maker in the region to participate in the program after it was introduced. Overall, the strategy worked well. This report provides a brief overview. As we shall see, while EXTRA benefited the fellows initially, their experience provided the regional transformation project with tools and motivation. It made our environments more productive and contributed to the development of a managerial culture that values facts and rigour in decision-making processes. There is still a lot of work ahead in order to fully capitalize on these achievements; the challenge of knowledge transfer is more than ever a key item on the agenda. In all, nine decision makers from Montérégie were members of the first four cohorts of the EXTRA program. Their intervention projects focused on system processes and procedures relevant to the Montérégie’s transformation. (A full list of fellows and their intervention projects is given in table 11.1). In preparing for this book, the fellows were consulted and asked for their personal experience with the program. Three topics were discussed: (1) expectations and skills gained, which was intended to identify what motivated the fellows to embark on the EXTRA program in the first place and what skills they developed by participating in it; (2) evidence-informed decision making, which sought to pinpoint the impact and the difficulties of using evidence in exercising a decision-making function; and (3) future prospects, which attempted to identify strategies or plans of action that could increase the use of evidence by decision makers in Montérégie.

170

What Leaders and Fellows Say

Table 11.1 Fellows and their intervention projects Year

Title

Project

2004

President and General Manager of the Agency

Development of, and systematic adjustments to, regional supervision of local service networks to maximize the supply of services and the impact on health and well-being using a population-based approach

2005

General Manager, CSSS

Set-up of an integrated service network: Reliance on evidence to organize the service offer, mainly for clients with chronic illnesses

Assistant General Manager, CSSS

Research on issues of concern to CSSS employees and physicians as part of the creation of the local service network

Public Health Director

Identifying and implementing strategies for developing the competencies required by local leaders in order to bring local public health plans to fruition

Director of CSSS Human Resources and Organizational Development

Action plan to retain employees and to develop and use competencies in order to reduce the impacts of labour shortages

Director of Front-line Services, General Services and Public Health

Reliance on evidence in health service organization by local service network players to carry out a clinical project

Coordinator of Physical Health, Emergency Services and Mental Health at the Agency

Winning strategies promoting the adoption of best practices in clinical/ administrative networks

Coordinator of Medical Affairs at the Agency

Trial of an agreed-upon method of collaboration between general practitioners and specialists for patient referral, with the goal of increased access to services

Assistant General Director of a Hospital

Reconfiguration of the service offer in mental health and geriatrics

2006

2007

Topic 1: Expectations and Skills Gained The managers who participated in the EXTRA program, all of them in mid-career when they applied to it, wanted to develop their scientific knowledge, both clinical and managerial. All of them felt that the program had enabled them to update their knowledge and acquire new knowledge so as to develop their ability to use research

The Experience in Montérégie

171

data. In the words of one participant: “I was keen to update my knowledge … I was curious to see how things had changed. And I find that they do a good job of that in the program.” And another: “That was my primary objective, and in that respect I was very well served.” Overall, the fellows achieved their objective of honing their skills in using research data in the management of their health care organizations. They talked about integrating the “lessons learned” from the program into their management practice; this usually means making it a more regular habit to consult evidence. Some participants also changed their behaviours or their ways of interacting with the health professionals with whom they dealt. “There are things that I have changed in the way I interact with the clinicians. I think that in that respect, the EXTRA program provides benefits. You can examine the evidence. That inspires you to look at things from an angle you had never considered before, to try to move forward in your thinking.” Some of the objectives proved more difficult to achieve, however, such as developing skills in playing an influential role towards local players (transformational leadership). While the majority of fellows felt that they had succeeded in this area as part of their project, some felt that they had not received all the tools necessary to accomplish this objective: “I know that I am able to influence people. But strategic influence is a domain unto itself. And using one’s influence to lead people to develop competencies is really something quite complex. I think I still have a long way to go in that area.” With respect to bringing researchers and managerial decision making closer together, the results are mixed. On one hand, some fellows described the successful collaboration established during the project: “I too would say that in my relationship with my mentor, I felt that he took a great interest in what I was bringing, because the world of research is a different universe, a bit disconnected from reality, and I was bringing the very practical, concrete experience of an institution. I consider that it was a very fruitful meeting.” On the other hand, some fellows expressed reservations in their attempt to develop collaborative relationships in their environments: “They tell us that it is very important, but they don’t tell us how we can go about it.” Finally, one of the main objectives pursued by the fellows and valued by the EXTRA program remains to be achieved. The managers who participated in the discussion regretted the lack of a network of Canadian contacts that could form a community of practice. For

172

What Leaders and Fellows Say

example: “I would have thought that it would be possible to create a community of practice around this.”

Topic 2: Evidence-Informed Decision Making The EXTRA program is intended to equip the directors, policymakers, and managers of the health care system with the skills to find relevant research work and make use of it as part of their daily professional activities. These managers could also act as agents of change in their environments. There is a general consensus that the EXTRA program promoted the introduction of real transformations in the Montérégie health and social services system. The fellows assert that without EXTRA it would have been extremely challenging, or even impossible, to achieve the changes that were needed for successful transformation. EXTRA has emphasized the importance of basing changes and new directions on evidence. This has encouraged many managers, even outside the program, to take pains to verify the extent to which their decisions are consistent with best practices. While the results of consulting evidence are certainly taking root in the individual practices of managers trained by the EXTRA program, some have found it difficult to play the role of an agent of change on a broader scale: “I am less successful. Action is needed. I don’t know what we can do in order to truly become an agent of change.” Nevertheless, some people feel that they are able to wield greater influence in their environments thanks to the use of evidence: “In terms of added value, the time I spent in the EXTRA program has taught me to present projects in a strategic manner and to ground them very solidly from a scientific point of view.” The managers who say they feel ill-equipped believe that the program did not examine aspects related to the emergence of transformational leadership in sufficient depth. They say they are lacking essential knowledge that would enable them to strategically influence their organizations: “I felt that this was not handled as it should have been, as they might have done for certain aspects of strategic change management. The module was short … the articles dense … I felt that this could have been given a bit more attention within the framework of the program.” In fact, the ability to make judicious use of evidence is an asset that helps managers to better exercise the leadership that is expected of them in their environment: leadership that builds its credibility

The Experience in Montérégie

173

on a foundation of familiarity with evidence and knowledge of how to use it in context in an appropriate and useful way. However, some of the feedback from the EXTRA fellows indicated that the program is rather “timid” on this point, without enough emphasis on introducing evidence in decision making. Others say that the subject comes too late in the order of things (module 4): “What I would say is that there may have been a problem in the sequence of things. It is when you are preparing your project that you need tools and strategies for wielding influence. That is when you should have them, so that you are able to use them. It is rather as if we had a theory that might not be useful in our jobs, whereas the thing that was most useful, we had that starting in the third residency.” Another fellow said, “Module 4 was a very strong point. I would like to have had it sooner.” In short, the fellows suggest that the program could be better adapted to their management role, particularly if they are expected to become influential agents of change in their environments. With this in mind, it would be advisable to review and expand certain program content, especially the reading of research data, taking into account the political, cultural, and organizational factors that influence decision making, as well as the role of an agent of change.

Evidence: Striking a Better Balance between Clinical and Management Data One of the things most frequently questioned by the fellows pertains to the objective of strengthening their research abilities. Those who were trained in the clinical area certainly perceive this as an opportunity to refresh their knowledge. However, the EXTRA program targets senior managers, and they question whether it is truly worthwhile for managers to acquire a uniform scientific foundation, fairly advanced to boot, and corresponding skills in the search for evidence. In short, the fellows feel that the training goes into too much detail, given their functions and the roles they play in their environments: “It seems to me that these are not tools that will be terribly useful as far as managerial action is concerned, and that is what I was looking for.” Another fellow said that “some of the knowledge was not useful to me, and I considered it superfluous to include it in this training process.” In contrast, the fellows’ increased awareness of the use of evidence gives them a real appetite for management data. But while clinical

174

What Leaders and Fellows Say

scientific data are abundant, managerial data are a much scarcer commodity: “As far as clinical data are concerned, I quickly found what I was looking for. But in the area of management data, I did not find much that was different, from which I could draw inspiration.” The virtual office is a case in point. Everyone acknowledges the wealth of clinical data it offers, but it is less well-stocked with management data: “The area where I found there was some weakness tended to be that of administrative data or literature.” While the EXTRA fellows recognize the importance of being familiar with the world of research and evidence, they also feel that the program should focus on making better use of information in the decision-making process. The program should therefore aim to develop fellows’ ability to use research evidence in their decision making. A Demanding Linkage with the World of Research The program offers fellows an opening into the world of research through the theoretical lessons learned during the various residencies and through the close ties that the program fosters between researchers and fellows. “I think it was quite positive,” said one. “The program enabled me to dig deeper with various researchers,” said another. However, the EXTRA experience apparently did not enable all the fellows to develop skills that would help them to re-establish this linkage once their project was over. The subjects that interest managers seem, according to some, to fall short of arousing the enthusiasm of researchers. As a result, managers find themselves unable to establish partnerships that would enable them to obtain data of great value to their organization: “The things that we need help with are not the things that interest people who are currently doing research. They tell us to stick close to what is being done in terms of research in order to avoid falling behind, but we don’t easily find help in that.” For many of the fellows, this linkage with the research community appears to be a key factor in enabling them to forge ahead in continuing to use evidence in their environment. Although this matter of a linkage was discussed during the training, the fellows were given a high-level overview, whereas they would have liked to learn more about ways to stimulate and arrange the partnerships they wished to establish. The desired balance between clinical and management data lies once again at the heart of the issue, since it is mainly the concern with evidence-informed management that the fellows would have liked to address further.

The Experience in Montérégie

175

An Opportunity for Networking The fellows who participated in the discussions in Montérégie all note, with regret, that the program did not give rise to any community of practice. Others lament that interaction between the fellows often takes place based on their professional affiliation, rather than on the management challenges that they encounter in the health care system. This approach leads the fellows to consider each other as players in different realms, rather than as a group of managers, which in turn makes it more difficult to establish cohesion within the group: “The very configuration of the group was based, not on the general common denominator of management, but instead on professional backgrounds. We have twelve nurses, four physicians, etc. … As a result, people in my group tended to cluster together according to their professional affiliation.” Crucial Support from the Organization EXTRA fellows unanimously agreed that without adequate support from their organizations, their projects would have been threatened. The fellow’s organization must be an “accomplice” in the project to (a) provide him or her with access to the resources that will be useful, or even essential, to the intervention project; and (b) increase the chances that the project will be integrated into the environment’s reality and become sustainable. As one fellow put it, “It takes strategies and new organizational methods to make this leadership possible. It is not the EXTRA candidate’s responsibility to become the champion of everything.” In Montérégie, this support varied from one group and one environment to another. The more established fellows attempted to take on the burden of work associated with the program all on their own, or nearly so: “What I came to realize as the project advanced is that I should have formed a team in the field to work with me, and just supply it with input, instead of bearing the burden of a project that I later had difficulty in bringing down to ground level because it was a bit too far removed from the reality of the people who worked in the field on a daily basis.” Recent fellows attached to the agency praised the knowledge brokers (who have been in place for about two years) as major contributors to their projects: “In my project, I was lucky enough to have a knowledge broker. I think I would not have been able to make progress without her. I think this is indispensable.” Some went so far as to

176

What Leaders and Fellows Say

suggest that future EXTRA fellows from the region should be more closely linked with the knowledge brokers because, they said, “the greatest danger that threatens the fellows is that of being isolated.” It is not just the execution of the project itself that would benefit from support; ongoing assistance would be useful, starting with the preparation of the application and ending with the dissemination of the results of the project. On this point, the fellows are unanimous in recognizing that the project results need to be shared and disseminated but that at present this does not happen: “The CHSRF encourages us to publish, and I think we need to go along with that. We need to do it.” In short, the organizational support that the fellows would wish for future EXTRA candidates needs to be unflagging and well structured in order for organizational and individual objectives to be achieved.

Topic 3: Future Prospects When asked to identify other strategies or actions applicable in Montérégie that would be likely to promote evidence-informed decision making, the EXTRA fellows made some interesting proposals. The fellows were convinced that the EXTRA program “is not an individual project, but an organizational project” and believed that if the CHSRF were to take an active role in clearly informing the organizations about the conditions needed for the success of the project, as well as about the expectations for its execution, the results would be significant. By being involved in the project in this way, the organization would be more inclined to contribute actively to its success by facilitating access to the resources that the fellows need, be it for preparation before the competition, for professional and technical support, or for the dissemination of results. In this way, the organization would take an interest in following the project and would contribute to disseminating the results appropriately. In brief, the entire organization could benefit from the effects of a single EXTRA grant if it were explicitly encouraged to be a part of it. Increasing the awareness of boards of directors about the importance of supporting evidence-informed projects and increasing the awareness of executive committees about the role that the organization should play towards the fellow are worthwhile initiatives in that direction. Deliberate actions drawing attention to the fellows and their projects could also prove beneficial; they would, among other things,

The Experience in Montérégie

177

bear witness to the importance that is attached to the use of evidence in the health and social services system. Presenting the results of their projects at symposiums or conferences, reporting on their experiences, and helping new cohorts of fellows to benefit in concrete ways are roles that the EXTRA fellows of Montérégie say they are prepared to play. Such actions could serve to promote the emergence of an evidence-informed culture and open the door to the development of new competencies in the use of evidence in organizational decision-making processes by other managers.

Conclusion At the mid-point of its planned path, EXTRA has already proven to be a powerful catalyst for change in developing its fellows, as well as in advancing the transformation strategy in our region. The program has directly contributed to strengthening leadership, making specific transformations possible at the local and regional levels. These success stories have shown the way and constitute a catalyst for change that promotes a cultural shift towards evidence-informed decision making. Most of the EXTRA achievements have directly influenced the group dynamics of the Montérégie network. For example, many local projects were subsequently replicated in other environments; the regional projects were drivers of similar initiatives throughout the regional system and at the supraregional level. Some EXTRA fellows have been invited to play an expanded role of influence across the system once they have completed their program. Everyone agrees that a linkage must be made between the universe of ideas and the world of organizations. The experience of the fellows in Montérégie shows that a major step has been taken in that direction. Further steps must follow if we truly aspire to narrow the gap between what we know and what we do in our field of activity. It is clear that developing the personal skills of managers is necessary. However, experience confirms that this must be accompanied by new organizational practices and a strengthening of incentives in terms of both management and research. On this perspective, the EXTRA program remains a preferred avenue and must continue to evolve. As for the agency, experience teaches us that we must pursue our efforts to exploit the full potential of this valuable resource for modernizing the Canadian health care system.

12 The Whole Elephant Many Perspectives, One Transformation at a Nova Scotia Health District chris power

We in health services are like the blind men and the elephant.1 The task of caring for our citizens and our communities is so enormous – like the elephant – that many of us who work at it can perceive only what is right in front of us. In doing so, we become quite convinced of the rightness of our perceptions. However, like the blind men of the fable, we all “see” something different. The part-of-the-elephant attitude diminishes our capacity to manage and deliver health care services. If we are to provide the best care possible for our patients, we cannot reduce the system to its parts without recognizing and appreciating its whole. This philosophy was clearly understood by each of the seven EXTRA fellows from Capital Health in Nova Scotia.2 Their intervention projects approached 1 The parable of the blind men and the elephant has been attributed to several groups. The poem by American John Godfrey Saxe (1816–87) is perhaps the version best known in the West. 2 Capital Health – formally known as the Capital District Health Authority – is the largest provider of health services in Nova Scotia. Through a network of hospitals, health centres, and community-based programs, its twelve thousand staff deliver medical and surgical care, mental health care, public and community health programs, addiction prevention and treatment, and environmental health to the four hundred thousand adult residents of Halifax Regional Municipality and part of an adjoining county. Capital Health also provides specialist services to the rest of Nova Scotia and to the other Atlantic provinces. In addition, it conducts health research and is a partner with Dalhousie University and the Nova Scotia Community College in providing academic and clinical learning experiences for physicians and other health care professionals.

Transformation at a Nova Scotia Health District

179

health care from different perspectives, but each was definite that his or her research would be conducted in the context of evidence-based improvements to the health care delivery system as a whole. From the technological angle, Dr Steven Soroka sought to implement components of an information management system to improve the care of patients with chronic kidney disease and thereby provide evidence of broader benefits. Kenneth Baird took an administrative perspective and examined how to improve organizational effectiveness through informed decision making. Susan E. Smith looked at planning for physician resources and developed an instrument and a process that linked clinical departments, helping them to consider their needs in terms of the whole organization. Dr Sam Campbell and Karen MacRury-Sweet examined different patient-flow processes, the former from a clinical perspective that aimed to improve a patient’s transition from primary to secondary care and the latter from an administrative viewpoint that addressed blockages to timely care through the emergency department. Mary Russell and Dr Brendan Carr stepped into the community with their projects: Ms Russell explored ways to measure the promotion of health, while Dr Carr explored ways to introduce novel communitybased collaborative practice teams. The EXTRA projects undertaken by Capital Health fellows, while separate and discrete, aimed to improve the overall management and delivery of services to the benefit of patients and their families. Whether focusing on the wider use of information technology, better organizational decision making and resource planning, improved processes in providing care, or renewed efforts on population health and community care, each participant undertook research that resulted in evidence relevant to other parts of the system. Each project sought improvements in the parts that connect to the whole. Each was fully aware that the entire elephant had to be cared for – and at the very time that it required a particularly high level of care.

Finding Our Way The value of the EXTRA program to Capital Health and to its participants on a personal level is being derived at a time of considerable challenge and transformation for the organization. In the years following its 2001 creation by the provincial government with the merger of three main entities – the region’s largest teaching hospital, a health board, and a psychiatric facility – many saw Capital Health

180

What Leaders and Fellows Say

struggling to find its way. As the organization grew, spending increased, health outcomes worsened, and wait times lengthened. Employee surveys indicated high dissatisfaction, and reports of staff bullying were worrisome. Many questioned whether an organization dedicated to caring for people had lost sight of its reason for existing. In March 2007, Capital Health’s leadership launched a process it called Strategic Quest to bring about fundamental change. It sought a shift from an illness model of health care to a wellness model – one in which citizens shared accountability with care providers for their personal health, as well as for the health of the system. To do this, Capital Health knew it would need to realign resources, and more importantly, it would need changes in attitudes and behaviours of health providers and citizens alike. To achieve this, Capital Health took an unconventional approach to its strategic planning. Research and public involvement were paramount. Over many months, thousands of different types of conversations took place – with stakeholders, community organizations, patients, families, staff, physicians, learners, and volunteers – in pursuit of a better understanding of Capital Health’s role and relationship within the community and the world. The overarching question was, “What would it take to create a world-leading haven for peoplecentred health, healing, and learning?” The process drew out some painful but necessary truths. “We uncovered deep wounds in the spirit of our people and a sharp hunger for a better way,” noted Our Promise, the resulting plan.3 “We heard the plea for a bigger tomorrow grow ever louder – a call for a tomorrow in which each and every one of us are awakened to our essential humanity and that of our fellow traveller.” How does this align with EXTRA? In a word, the answer is, perfectly. Although the executive training program predates Capital Health’s Strategic Quest, the two efforts share a passion for improving decision making in the health care system. Both are committed to developing capacity and leadership in optimizing the use of research evidence; indeed, the EXTRA program has reinforced Capital Health’s innovative approach to leadership development in support of its cultural transformation.4 Among the essentials of the transformation is permission to take risks and embrace tension, a belief 3 For the complete plan, visit www.ourpromise.ca. 4 For more information, visit Capital Health’s website www.cdha.nshealth. ca, click on Capital Health A–Z, and view Leadershift.

Transformation at a Nova Scotia Health District

181

that different outcomes require different processes, and an invitation to feed the hunger to learn and discover – elements that show up, in varying degrees, in all the EXTRA projects undertaken by Capital Health’s fellows.

Silos? What Silos? How often in his years of clinical practice had Dr Sam Campbell found himself thinking, “There must be a better way to do this?” As director of continuous quality improvement in the department of emergency medicine at Capital Health’s Queen Elizabeth II Health Sciences Centre, he noted in particular the challenge of moving patients from the care of community caregivers to secondary and tertiary care hospitals. What were the obstacles, and how could we remove them? Dr Campbell embraced the opportunity offered by the EXTRA program and as part of the 2005 cohort sought a “better way” to reduce the barriers to care and communication faced by a patient moving from primary to secondary care. He chose deep vein thrombosis (DVT) as an example of a clinical presentation that demonstrated these barriers. Dr Campbell led a team that adapted a diagnostic pathway that enabled family physicians to conduct a step-by-step assessment of their patients. Built on new evidence-based protocols for diagnosis and treatment, the tool allowed doctors to determine the probability of the condition. As a result, family physicians were able to diagnose and treat most patients with suspected DVT. For those requiring referral, the pathway eased their transition into the hospital system. The process involved advanced-care paramedics, emergency physicians, and radiology and hematology staff. And at the centre of the process, and receiving better care, was the patient. At the same time, the project team recognized it would have a greater chance for success if the process could be shown to make life easier for the caregivers. To that end, the team was careful not to be defensive and instead embraced the “productivity of resistance,” a concept that suggests resistance can be constructive in highlighting problems. The team acknowledged the tension and asked staff and stakeholders for feedback in a highly visible way: by encouraging them to write suggestions on a poster displayed in the emergency department. The process evolved and improved, and everyone involved felt they had input and could support the effort.

182

What Leaders and Fellows Say

A year into the project, family physicians using the pathway rated it 8.99 out of 10. Ninety-five percent of emergency physicians were satisfied or very satisfied with it, as were 89 percent of the advancedcare paramedics. In addition, 95 percent of the patients contacted were satisfied or very satisfied with their care. Patients referred to emergency saw their length of stay decrease by more than an hour and a half. The pathway is now considered the standard of care for DVT, and similar approaches are being developed in other areas such as anticoagulation management. For too long, it seemed, Dr Campbell had observed caregivers functioning in their primary- and secondary-care silos, often to the detriment of patients. He was determined to show, through research, that if caregivers were going to be in a silo, they were better off being in the patient’s silo – together. Shattering silos and taking the risk entailed in doing so was also clearly illustrated in another EXTRA project. As part of the first cohort of fellows, Susan E. Smith, then director of medical services at Capital Health, sought to make medium- and long-term planning for physician resources into a process that would be informed by evidence and that would consider a wider impact within the organization. Before her work, individual clinical departments planned for new physicians without considering costs, including support staff and space, or the potential effect on other departments. Her initiative developed an instrument that asked each clinical discipline at Capital Health to identify its additional physician resources based on evidence of need five years out. The information was consolidated, and clinical chiefs gathered to examine and discuss all the various parts. These types of conversations had not previously occurred. As Smith noted after the first of what has since become an annual process, the chiefs found the conversations valuable. By coming together, they were able to consider their needs beyond their individual departments and look at their requests in terms of the impact on other departments and the whole organization. The project wasn’t a straight line to patient care – a focus at the very heart of Capital Health’s cultural transformation – but Smith’s use of her research evidence to improve the organization’s physician resource planning would translate in due course to the patient receiving medical care from the right person at the right time. Karen MacRury-Sweet was looking for a way to push her own personal envelope when she applied for the EXTRA program. She

Transformation at a Nova Scotia Health District

183

deliberately designed her project, which involved assessing patient flow in the emergency department and developing a “patient dashboard,” to be outside her comfort zone so she would be forced to learn new skills. She discovered, however, that the biggest benefit of the program was learning that no person, no problem or no department was unique. “Every one of us across the country is dealing with the same issues,” she said. “The networking has been the best aspect – helping us realize that no one is an island.” The networking extended to others closer to home as well. Her project brought her into contact with professionals within Capital Health with whom she would not normally have worked and who shared her interest in creating a predicative model to forecast patient flow in emergency. The sharing of knowledge across departments, despite silos, benefitted her project, her day-to-day work, and, ultimately, the patients of Capital Health.

Beating a New Path The belief that a different outcome requires a different process underpinned Capital Health’s Strategic Quest. Similarly, it drove a number of projects by EXTRA fellows from the organization, such as Campbell’s DVT pathway, Smith’s physician resource planning process, and MacRury-Sweet’s patient dashboard. Dr Steven Soroka, director of the hemodialysis and predialysis program at Capital Health, was another who sought a different outcome through a different process. Soroka’s approach was broader use of information technology. Even as a strong reliance on paper systems persisted in Nova Scotia’s health care system, he saw many benefits to building a better information management system: improved day-to-day care and longterm care planning for patients with chronic kidney disease; higher-quality data for more informed decisions on expanding programs or developing new ones; and increased capacity for clinical research to increase the knowledge available in the literature. Soroka’s search for the training that would help him build this system coincided with the launch of the EXTRA program in 2004. His project began with Capital Health’s renal program and eventually grew to include all renal programs in Nova Scotia. One key to success discovered early in the project was the importance of finding champions within the allied health professionals and physician

184

What Leaders and Fellows Say

population. Their enthusiasm for the technology encouraged others to adopt the new system. Today, a half-decade later – a lifetime in technological advancement – there is an acceptance and understanding that an integrated information management system is the right tool for health care providers, as well as for administrators and researchers. As Capital Health transforms and old ways are challenged, its IT department is developing a similar system for use in other endeavours beyond the renal program. As well, there is an opportunity to develop a system to collect information on specific indicators that would help renal programs across Nova Scotia improve care. An integrated information management system could give the government a heads-up about future need for services in specific areas of the province. Dr Brendan Carr similarly sought a new process in the search for a different outcome for the individuals and families served by Capital Health. As vice-president of medicine at Capital Health, Carr still pulled one shift a week in the emergency department. There, he regularly saw patients entering that part of the health care system with needs that could have been prevented or cared for through wellness programming, lifestyle education, and supports based within their communities. Carr’s EXTRA goal is a strategy that promotes integration between health care services and community-based resources. The desired outcome is a people-centred community health care experience that responds to the social determinants of health. To that end, work on creating community health teams in two communities began in the spring of 2009. While this different process is still to be proven, what is certain is that the old process would have continued to produce old outcomes. Carr counted as the biggest success of his project its alignment with Capital Health’s vision to become a world-leading haven for people-centred health, healing, and learning. As the organization transformed, people were more ready to embrace the outcomes of his research. Having these two aspects closely related meant his project had greater impact than he initially anticipated across the organization. “More importantly,” he points out, “the evidence and concepts that informed my project will endure in our organization and affect many other important initiatives.” Also casting an eye beyond the hospital system, Mary Russell was a firm believer that the way to improve the health of Nova Scotians was

Transformation at a Nova Scotia Health District

185

not through more funding for acute care – an increasingly unlikely proposition – but by going upstream and focusing on what it was in people’s lives that determined their level of health. As Capital Health shifted to a wellness model, traditional acute-care indicators like wait times, medication errors, and infection rates, would be insufficient. As part of her EXTRA project, Russell and her team researched and refined a list of community health indicators, identifying one that required priority attention: physical activity. To address this, a community-wide strategy was developed to improve rates of physical activity throughout Capital Health’s population. Among the different outcomes Russell credited to the project was the creation of non-traditional partnerships in which all partners accepted, as shared responsibility, their role in addressing physical inactivity in the community; in other words, the issue was not a sole responsibility of the health care sector. The partnership included a school board, a health charity, a university, different levels of government, and a children and women’s health centre. This new type of partnership – and the concept and philosophy of shared responsibility – may well be a model used to address other issues that affect health.

Hungry to Learn As Capital Health undergoes a cultural transformation, it does so with a deep commitment to fostering a community of learners. A key part of the undertaking is to shift to a new way of thinking, believing, feeling, and doing. In this, Capital Health has found alignment with the EXTRA program. To a person, participants report a change in professional perspective, significant learning, and in some cases a deeper understanding of themselves. As a member of the leadership team at Capital Health, Kenneth Baird felt he had a responsibility to keep himself at the forefront of developments in the health care system and to bring those developments back to his organization. The vice-president of diagnostic, therapeutic, and facilities support, he found EXTRA to be a “transformational” program that helped him improve the quality of nonclinical decision making at Capital Health and at the same time fulfill his own desire for personal growth. Baird’s project resulted in a framework for administrators to apply research to their decision making, with Capital Health’s EXTRA fellows brought together in a steering committee as part of

186

What Leaders and Fellows Say

this framework. He learned along the way that organizational readiness would be fundamental to his project’s success. Great effort had to be taken to ensure that people and processes were ready to move in the new direction. The pace of the move could not be too fast. He also learned that evidence alone cannot and does not support a good decision making process. “Evidence can and should be used by leaders in health care organizations to inform their decisions,” he said, “but it must always be considered in the context or environment – both internal and external to the organization – within which it is being applied.” He was amazed at the amount of knowledge and expertise available at his fingertips. “Prior to participating in EXTRA, health services research was always a bit of a mystery to me, and an area of work that I previously felt had minimal relevance in my day-to-day world,” he said. EXTRA opened his eyes to the expanse of information available to help him make better-quality decisions. Ms MacRury-Sweet said her EXTRA experience profoundly changed every aspect of her work. “I look at things differently, read things differently, and address things that I probably would not have looked for previously,” she said. “It has improved my future and enabled me to influence long-term decisions while working with evidence. I take every opportunity now to incorporate research into my decision making. It takes time, but it is worth it.” Dr Campbell, today also site chief of emergency at the health sciences centre, echoed his colleague. He said he uses his EXTRA skills every day in one way or another. “I endeavor to insist that we look for and appraise existing evidence, as well as gather our own where possible in all initiatives that I am involved in.” Dr Carr, who initiated a collaborative process that led to the community health teams, said he was particularly pleased to be able to enhance his ability to use health services research in his administrative decision making. “These skills are not well covered in professional school or in many graduate programs,” he said. “They are, in my opinion, a critical requirement to ensure the sustainability and future quality of the Canadian health care system. They need to be considered among our core competencies as leaders.”

The Elephant Is Big From the inception of EXTRA, we at Capital Health saw the value of having its members exposed to new ways of thinking, new colleagues,

Transformation at a Nova Scotia Health District

187

and new learning. The importance of this approach continued to grow as the organization launched its transformation strategy and committed to our patients and our communities to find new ways of coming together and providing care. The rationale and goals of the EXTRA program have aligned with our strategic efforts and attracted participants who would help lead our transformation. Elements of several projects have served as prime examples of the Our Promise strategy in action. Not surprisingly, all the initial projects undertaken by Capital Health fellows have continued within the organization. What is particularly exceptional is that the impact of the projects has extended far beyond the fellows’ specific area of research. Their influence has grown beyond the individual projects themselves. The efforts of a few have inspired countless others – to open their hearts and minds to change, to continued learning, to the possibility of different outcomes, and to the importance of informed decision making. We are mindful, however, that such sustained influence requires diligence and effort. Busy schedules, old ways, and new demands can dilute any longer-term impact that EXTRA and its participants seek. As Soroka expressed it, while the program offered him an extraordinary opportunity to engage in dialogue with others, he is disappointed that his hoped-for community of practice has had limited success. “There have been several activities that brought the EXTRA cohorts together, but this seems to have waned over the last year,” he said. At Capital Health, we can strive to do better in this area. Baird’s steering committee of EXTRA fellows is a start to using their knowledge and talents more broadly for the organization. As we work to strengthen our learning networks, we need more champions and advocates. EXTRA’s success for Capital Health participants has made the program a much sought-after experience, and each year we face the difficult task of deciding which applicants to support. Given the small cohort annually – and the size of the health care system we are seeking to influence, let alone our individual organizations – we need to better leverage participants’ learning experience and create new opportunities. Capital Health has set itself on an intense cultural transformation. We know we need to do better, and we are determined to do so. As we find ourselves on a journey to become a world-leading haven for people-centred health, healing, and learning, our success will depend greatly on our capacity to take a broad view of the system in which we work and in which we are privileged to provide care to our fellow

188

What Leaders and Fellows Say

citizens. With its focus on improving its use of evidence in decision making, the EXTRA program has been invaluable in enabling Capital Health to expand its capacity to see – and understand – all parts of the elephant as integral to one larger being.

Conclusion Supporting the Individual as Change Agent and the Organization as Responsive to Change ward flemons , andrea seymour, and carl taillon

Engaging the Chief Executive in Cultural Change Ward Flemons The EXTRA program presents interesting opportunities and challenges for the parties that enter into a “tripartite” agreement for a twoyear investment of time and energy. Each party – the fellows, their organizations, and the Canadian Health Services Research Foundation (CHSRF) – may have different objectives and expectations for this investment. For fellows, the objective is education; the program offers an incredible experience to learn, in a relatively short period, how to systematically find, appraise, and apply research evidence in health services management decisions. When the EXTRA program was brought to my attention by a senior member of our health care organization – the then Calgary Health Region in Alberta – I was intrigued and interested in the program’s focus on using researchbased evidence for decision making, given my previous pursuits in clinical-outcomes research and interests in organization-wide quality improvement and patient safety efforts. My main objective was to acquire additional training on how to become a more effective organizational leader. Instinctively, one might think that the main objective for the sponsoring organization is complementary to that of the fellows, since EXTRA offers an effective leadership development program for an

190

What Leaders and Fellows Say

organization’s senior health administrators. Arguably though, the primary objective is tied to the experiential learning component of the program, where fellows address a local need, issue, or problem within their work and organization by undertaking an intervention project. This change management project presents a second objective of EXTRA: the possibility for the sponsoring organization to achieve needed change in an area of opportunity that is important to them. It wasn’t until I had been accepted into EXTRA and started in the first training module that I more fully appreciated a third objective of the program: for CHSRF to shift the culture of sponsoring organizations towards being more systematic in the way that research-based evidence is used in policy decision making. Although professional development of the fellow is a clearly stated objective of EXTRA, I don’t believe my sponsoring organization completely embraced the CHSRF’s overarching objective to improve the extent to which research is used to inform decision making. As with any change, the first step is acknowledging the need for improvement. Although the CHSRF made admirable attempts to make the case for change and assess organizational readiness – for example, by equipping organizations with a self-assessment tool and discussion guide to examine their capacity to gather, interpret, and use research in making health services decisions (CHSRF 2005)- neither they nor I were particularly successful at convincing my organization to shift its commitment towards an increased reliance on research-based decision making. Successfully shifting an organization’s culture depends on three important factors. First is the degree of understanding and commitment that an organization’s chief executive has for the CHSRF objectives before asking a member of his or her organization to apply for an EXTRA fellowship. Second is the seniority of the EXTRA fellow within his or her organization, that is, the likelihood that the fellow will have enough regular contact with the executive to influence how decisions get made. The third factor is the stability of the organization’s executive structure. During the final year of my EXTRA fellowship, I experienced significant change in my reporting relationship to senior executives, which greatly reduced my opportunities to interact with them. In addition, shortly after I completed the fellowship, the provincial ministry of health in Alberta amalgamated all of its health care regions in the province under the Alberta Health Services, eliminating in the process the senior executive infrastructure

Supporting the Individual as Change Agent

191

that was in place at the Calgary Health Region when I had begun the fellowship two years earlier. With this structural reform, the opportunities to contribute substantially to a shift in the culture of the newly formed organization have been limited significantly. During the EXTRA program I became more aware that the organization’s major focus was not my executive-level training but, rather, the deliverables from my intervention project, which aimed to improve patient flow in emergency departments (CHSRF 2008). The expectation developed that my project would deliver system-wide change over a relatively short time frame. Managing this expectation was challenging and not altogether successful. Although the intervention project succeeded in many places and at many levels, sustained system-level results were not realized; thus, the organization’s prime objective for the intervention project was, from its point of view, not met. Organizations that have made the EXTRA program part of their strategic plan, both for leadership development and for learning how to access and use research evidence, are likely to realize more recognizable change in the degree to which they embrace evidenceinformed decision making. Executive leaders of health care organizations need to understand two important things. First, improvement takes time, and making large system change requires a sustained effort from many parts of the organization. It is unrealistic to expect that a single project, especially one undertaken as part of an EXTRA fellow’s learning experience, will result in a fundamental shift in the way that business is conducted. Second, what the EXTRA program is attempting to influence is organizational culture – “how things are done around here, how decisions are made around here.” However, culture is shifted neither by a single project nor by an individual. For the organization and the EXTRA fellow to be successful in the long term requires a clear corporate vision for improvement and an understanding of how the EXTRA fellowship, specifically the intervention project, plays a role within that vision. Shifts in culture come over extended periods of time and through repeated refinements and experience with doing things differently. Consequently, the CHSRF and the organizations they are trying to influence may need to be more explicit in articulating their objectives. If the goal is to introduce a process of evidence-informed decision making, then there should be a commitment to building an infrastructure to support it. At the very least, there should be a committee within the organization that is led

192

What Leaders and Fellows Say

by someone from the “C-suite” – if not the chief executive officer, then the chief operating officer. Such a committee would require the support and buy-in of other organizational decision makers and should include the EXTRA fellow(s) that the organization has sponsored. The mandate of such a committee should be to experiment with different ways to determine the appropriate questions that would lend themselves to using research evidence to assist in formulating answers or solutions and to identifying a process to collect relevant information and to synthesize it in a way that makes it accessible to decision makers. Finally, an evaluation plan is needed, informed by meta-evidence, that demonstrates such a group has had a positive impact on the management and delivery of health care. All of this would certainly require an additional time commitment from EXTRA fellows and their sponsoring organizations. I would predict, however, that it would increase the likelihood that organizations will adopt a more research-informed approach to decision making, allowing the CHSRF’s vision to be more fully realized.

Embedding the Learning Andrea Seymour A lot has changed since I graduated from the EXTRA program in August 2007. Most significant for me is that I have transitioned to a career outside the health field, as assistant deputy minister, Department of supply and services, for the government of New Brunswick. Most significant for the province is that the health system has undergone a complete overhaul – collapsing eight regional health authorities into two, a change that took full effect in September 2008. With this restructuring, the regional health authority, River Valley Health, and the senior management team with whom I once worked no longer exist. All but one member of our senior management team left the newly created regional health authority. Unfortunately, with the exodus of senior management team commitment for the intervention project languished during the transition period, and the program was eventually discontinued in late 2009. Concurrently, however, a Healthy Workplace Strategy was introduced within Part I of the Government of New Brunswick, and staff from the regional health authority moved on to implement the program within the civil service of New Brunswick. My intervention project was designed to develop an organizationwide healthy-workplace governance structure, to assess, through a

Supporting the Individual as Change Agent

193

combination of approaches, the health risks of its employees, and to prepare for discussion a recommended approach for implementation (Seymour and Dupré 2007, 2008). To inform this work and to help us understand the health risks that employees face, a comprehensive health risk assessment was undertaken across the organization. The assessment consisted of two parts: first, physical testing of staff for blood glucose, cholesterol, blood pressure, weight, height, and body mass index; and second, an on-line staff questionnaire that assessed workplace culture, individual health practice, and environmental impacts on physical health (Seymour and Dupré 2007, 2008). The results of the assessment prompted the organization to develop a healthy-workplace policy, encompassing three pillars – organizational health (culture), personal health (practice), and musculo-skeletal health (environment). All of the necessary ingredients came together at the outset and throughout the project to lead to development of the policy. To start with, the project was supported by an organizationwide appeal that came from the chief executive’s office in 2006 calling for staff participation in a Healthy Workplace Steering Committee. The EXTRA program equipped me with the time and the research-use skills needed for the project to find its roots and take shape. This project was also in line with the organization’s mission, values, and strategic direction, Strong People and Strong Partnerships. With all of these factors in place, the creation of the policy simply aligned the organization’s commitment to a healthy and safe workplace with an imperative to embed health-related activities into the daily practice of the organization. (Seymour and Dupré 2007, 2008) The policy has since proven to be a critical, foundational element in supporting a change in mindset, creating lasting buy-in and securing the sustainability of the initiative. Despite the creation of the new health structure in New Brunswick and the departure of the River Valley Health (RVH) senior management team, RVH continued healthy workplace activities under the new structure and leadership. Evidence of this commitment was seen in October of 2008, when the Healthy Workplace Steering Committee put in place a calendar of events to celebrate Healthy Workplace week, including programming that extended far into 2009. In leading and sustaining an evidence-informed policy and practice around healthy workplaces, the fellowship experience was surely a success. But despite this triumph, a significant opportunity was lost. At the time that I was enrolled as an EXTRA fellow, few in my

194

What Leaders and Fellows Say

organization, apart from those intimately involved, would have made the connection that my intervention project was supported through EXTRA and the CHSRF. Even fewer, including those intimately involved, would have seen this project as a demonstration of the fundamental principles of evidence-informed decision making. In fact, even the research evidence that informed the shape and design of this project was ultimately superseded by the policy and activities that came after. Now, under a drastically new governance structure and with a complete turnover of the senior management team, there is little hope that this project has, in any way, brought about a culture of evidence-informed decision making. Looking back, it seems that there is a simple and manageable way of establishing a connection between intervention project work and the larger goal of breeding a culture of evidence-informed decision making in sponsor organizations. My suggestion comes from a simple observation: every time that I or someone from my team shared details of the healthy workplace policy and activities, it was, in fact, a missed opportunity. The presentations and discussions were ripe for communicating the basic and underlying value of evidenceinformed decision making. Perhaps the time is ripe, therefore, for the CHSRF to elicit more tangible support at the organizational level by requiring fellows to present to their board of directors, their senior management team, and their medical advisory committee the results of their intervention project, including how it affected practices within the home organization, and potential impacts external on health. If every fellow endeavoured to portray his or her project as a demonstration of what can be achieved when research is sought and applied to practice, then incrementally we would be able to effect real cultural change in the way that we think about and use the evidence and best practices that are available. Perhaps then we would truly begin to embed the learning in our organizations and throughout the Canadian health care system.

Supporting a College of Fellows Carl Taillon There is no denying that EXTRA training encourages more methodical and efficient management, an appreciation for the quality and value of conclusive research evidence, the use of such evidence in context, reflection about managerial issues with a view to exercising

Supporting the Individual as Change Agent

195

leadership, and long discussions and dialogue about change processes. All of this is crystallized in a practical intervention project firmly anchored in an activity involving the sponsoring organization as a whole. While all of this training is thorough, it nevertheless does little to prepare fellows for the unexpected challenges that await them after graduation. For many graduates, for example, the organizational environment that initially hosted their learning may have been turned upside down or even eliminated altogether owing to drastic structural reform. This was the experience of my co-authors, Ward Flemons and Andrea Seymour. I count myself lucky that this was not the case for me. I am, it seems, fortunate to have been able to continue my EXTRA activities by following a relatively straightforward career path after completing my EXTRA fellowship. Having said that, with one or two exceptions the management team of my institution, the Centre hospitalier universitaire de Québec (CHUQ), has been revamped since I started my activities in 2005. In my organization in 2005 we were getting started on a renewal of our health care management model, moving from a “silo” model to a “network” model in hopes of breaking down the barriers between the various health care disciplines to promote better integration of care. The need for renewal work stemmed from a hospital amalgamation in the mid-1990s that had aimed to create a university hospital but that had ultimately failed to reconcile the individual institutional cultures, leading to a breakdown of the medical-administrative structure. At the time, the discouraged medical staff no longer exercised any self-regulatory control over medical practice. To help address these issues, my organization supported my EXTRA fellowship and intervention project to rebuild the relationship between the organization’s management team (especially managing physicians) and medical staff (Taillon 2007). Rebuilding these relationships is taking longer than the two-year EXTRA fellowship allowed. Fortunately, I have been able to continue this change management work as part of an ongoing organizational effort to develop health care policies and strategic directions conducive to the adoption of new, more efficient health care management models. Thus, the bridge that makes it possible to tie the EXTRA project to medium- or long-term prospects does exist; however, a significant barrier remains – the eternal state of affairs, the school of daily reality – that stands in the way of a fully successful implementation.

196

What Leaders and Fellows Say

Managers are constantly confronted, even harassed, by their daily realities, rigidly constrained to focus their attention on keeping average hospital stays as short as possible, closely monitoring the amount of time patients are spending on stretchers in emergency rooms, and managing wait times for various types of surgery – all of which must be reported to central authorities. In short, these “micro-management” activities greatly overshadow the horizons beyond the EXTRA project We must not underestimate the fellow’s responsibility in reaching for these horizons. The fellow has primary responsibility for using conclusive evidence as a way of getting all clinical professionals to participate in improving care for patients. For my part, as an EXTRA graduate, I am considering implementing two strategies to promote the achievement of that objective in my organization. First, we (my colleagues and I) are strengthening the joint participation of all clinicians in the continuous improvement of the quality of care and services. Accreditation Canada – an independent, not-for-profit organization that provides programs and guidance to health care organizations to help them improve the services they provide to their patients and clients based on standards of excellence – offers an opportunity in this regard that I believe is too often neglected. However, the efforts associated with the various standards, particularly those now identified as exemplary or leading practices, must be translated into concrete actions. Second, we must give some thought to addressing a challenge that I believe is crucial to the success of any continuous improvement endeavour: that of a professional attitude, particularly on the part of physicians. In fact, faced with all these efforts to “decompartmentalize” and integrate health care and service processes, the medical profession appears to be divided, even fragmented. This has consequences with respect to internal reactions, since the members feel that the changes have little to do with them. The medical profession seems to have little interest in contributing to a more effective way of dealing with the ever-growing complexity of the health care system. The profession has frequently settled for presenting a position based on the doctor-patient relationship alone, without taking into account the many institutions and players that make up the mosaic of the health care system as a whole. The Canadian Medical Association (the national medical professional association), in fact, has been considering the issue of physicians’ professional attitudes for some time now.

Supporting the Individual as Change Agent

197

From all the above, there emerges a wish or, rather, a suggestion. Perhaps the EXTRA program or the CHSRF could formally promote the enrolment of graduate fellows in an undertaking of national organizations that would be able to support the them beyond their EXTRA training or intervention project. I am aware that fellows can always continue these efforts in their own work, but I believe that a “formal enrolment” promoted by the CHSRF would foster a more serious commitment and concerted attempt by fellows and their sponsoring organizations to an ongoing process that would extend beyond the EXTRA fellowship training. I would like to share a small but nevertheless significant experience that supplemented my own EXTRA 2005–7 training. Just shy of graduating from the program, I was given an opportunity to offer mentoring services to a 2007–9 fellow and CHUQ colleague. I now realize that besides keeping the “course” of the program current, this mentoring exercise, which is closely related to the principles and practices of knowledge transfer and exchange, encouraged me to think about strategies for sustaining the overall experience of the EXTRA project. As a result of the mentoring relationship, my “professional” reading is now more firmly dedicated to the process of acquiring, understanding, and criticizing the conclusive evidence commonly shared in the literature. Arguably, the CHSRF could replicate my mentoring experience with other EXTRA graduates. It is but a short step for the CHSRF to identify, through a formal assessment and evaluation process, suitable candidates from its previous cohorts to play a coaching and mentoring role for up-and-coming fellows. Arguably, this type of mentorship could lead to greater solidarity among senior health system leaders. The result might be a “college of fellows” who could support each other in supporting a more evidence-informed health care system – the ultimate goal for the CHSRF.

Concluding reflections on the fellows commentary from the editors The EXTRA program has set out lofty goals for fellows, their host organizations, and the Canadian health system at large. But does the program live up to its large-scale expectations? And how can the program develop and advance to keep pace with the needs of health services executives?

198

What Leaders and Fellows Say

The complementary but distinct examples in this chapter from graduates reveal some common threads that we might well take to heart in refining the curriculum: the need to more closely bind sponsoring organizations – specifically, their assignment of time, attention, and priorities – to the goals of the EXTRA program. Dr Flemons challenges the CHSRF to more intimately involve the senior executive machinery, and thereby the strategic directions, of sponsoring organizations to its goals of catalyzing evidence-informed decision making. This reflection also bears directly on the question of team commitments to the fellowship, which in principle require a higher level of organizational commitment and capital. Meanwhile, Ms Seymour challenges fellows and their host organizations to adopt a holistic view of the EXTRA fellowship, positioning the intervention project work of fellows as a demonstration of the sound fundamental principles of evidence-informed decision making. Finally, Dr Taillon proposes a strategy to complement the EXTRA program and bring solidarity among fellows by supporting a college of fellows – an ongoing network to help create cultural change in organizations by predisposing them to evidence use, performance, and quality improvement as collaborating agents in health care system transformation. Despites its limits, EXTRA has showed success in training senior leaders to use research evidence more actively in the day-to-day financing, management, and delivery of health care. Dr Flemons, Dr Taillon, and Ms Seymour acknowledge successful outputs from their intervention project work. For the EXTRA program and the fellows’ intervention projects to yield lasting and systemic results, though, the chief executive and quite possibly the boards of sponsoring organizations will need to consider capacity building through programs like EXTRA as crucial to preparing for the challenges of tomorrow. An additional lesson for curriculum redevelopment may be the need for a more flexible and modular approach to the curriculum to enlarge the commitment and ongoing participation of senior and mid-level actors from host organizations, to increase the density of trained individuals within organisations. For its part, EXTRA can help prepare senior physician, nurse, and other health executives to behave as change agents in their organizations: to single out initiatives in their environment that need to be successfully advanced through competent management leadership rooted in evidence-informed management. Another lesson here might be to ensure a careful selection of the organizational change

Supporting the Individual as Change Agent

199

project to ensure senior commitment but also the feasibility of execution. Part of this feasibility assessment will be the range of hands on mentoring and program management supports that fellows will need to conclude their projects in a timely fashion. It is our view that the training curriculum can and must adjust its offerings to meet the aspirations of our fellows and the changing organisations in which they work. To create evidence-based leaders for tomorrow, we must act on their feedback and the changing imperatives we face to improve quality and performance, all rooted in the competent extraction, organisation, and use of evidence by the leadership cadre in our health care system.

Reference Canadian Health Services Research Foundation. 2008. Dispersing the crowds: How a health region is being guided by evidence and theory to chip away at emergency department overcrowding. http://www.chsrf.ca/ other_documents/pdf/WEB_11421_CHSRF_pp19_e_FINAL.pdf (Accessed 3 February 2009). Canadian Health Services Research Foundation. 2005. Is research working for you? A self-assessment tool and discussion guide for health services management and policy organizations. www.chsrf.ca (Accessed February 3, 2009) Seymour, Andrea, and Kathryne E. Dupré. 2007. An organizational intervention aimed at improving employee health. Healthcare Management forum 20(3): 10–13. – 2008. Advancing employee engagement through a healthy workplace strategy. Journal of Health Services Research & Policy 13(Supplement 1): 35–40. Taillon, Carl (propos recueillis par Ghislain Labelle). 2007. Effort de mobilisation médicale. Le Point (winter, edition Spéciale): 12–13.

This page intentionally left blank

Contributors

Editors jean-louis denis original extra academic coordinator and curriculum designer Lead Faculty (Module 5: 200611). Professeur titulaire et titulaire de la Chaire de recherche du Canada sur la gouvernance et la transformation des organisations et systèmes de santé, École nationale d’administration publique et chercheur à l’Institut de recherche en santé publique, Université de Montréal, Montreal, Quebec terrence sullivan current extra academic coordinator and lead faculty (Module 3: 2005–11); president and chief executive officer, Cancer Care Ontario, Toronto, Ontario

Other Contributors Owen Adams assistant secretary general, Research Policy and Planning, Canadian Medical Association, Ottawa, Ontario Malcolm Anderson assistant professor. Faculty of Health Sciences, Department of Physical Medicine and Rehabilitation, Queen’s University, Kingston, Ontario Lynda Atack Faculty of Nursing, Centennial College

202

Contributors

Robert Bell president and chief executive officer, University Health Network, Toronto, Ontario Sylvie Cantin agent de planification et organisation, Régie régionale de la santé et des services sociaux de la Montérégie, Longueuil, Quebec Susan Donaldson Susan Donaldson and Associates Ward Flemons professor of Medicine, vice president, Quality, Safety and Health Information for the Calgary Health Region, 2004; extra graduate 2008 Dorothy Forbes Arthur Labatt Family School of Nursing, University of Western Ontario Paula Goering director, Health Systems Research and Consulting Unit, Centre for Addiction & Mental Health, Toronto, Ontario; extra guest faculty (Module 2: 2004–8) Karen Golden-Biddle professor/Everett W. Lord Distinguished Faculty Scholar, Organizational Behavior Department, Boston University School of Management, Boston, Massachusetts; extra lead faculty (Module 4: 2005–7) Jeffrey S. Hoch health economist and research scientist, Health Policy, Management and Evaluation, Centre for Research on Inner City Health, University of Toronto, Toronto, Ontario; extra guest faculty (Module 2: 2004–8) Paul Lamarche professeur titulaire, Département d’administration de la santé, Université de Montréal, Groupe de recherche interdisciplinaire en santé (GRIS), Montreal, Quebec Ann Langley professeure, Service de l’enseignement du management/Department of Management, École des hautes études commerciales, Montréal, Québec; extra Guest Faculty (Module 4: 2005–7) John N. Lavis Canada Research Chair in Knowledge Transfer and Uptake, associate professor, Department of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Ontario; extra Lead Faculty (Module 1: 2004–8) Melanie Lavoie-Tremblay assistant professor, Faculty of Medicine, School of Nursing, McGill University, Montreal, Quebec

Contributors

203

Manon Lemonde associate professor, Faculty of Health Sciences, University of Ontario Institute of Technology, Oshawa, Ontario Jonathan Lomas founding chief executive officer, Canadian Health Services Research Foundation Margo Orchard senior consultant, Ministry of Health and Long Term Care, Ontario Raynald Pineault professor, Social and Preventive Medicine, University of Montreal, Montreal, Quebec Brian D. Postl dean, Faculty of Medicine, University of Manitoba; former president and chief executive officer, Winnipeg Regional Health Authority, Winnipeg, Manitoba Christine Power president and chief executive officer, Capital District Health Authority, Halifax, Nova Scotia Trish Reay assistant professor, Faculty of Business, Strategic Management and Organization, University of Alberta, Edmonton, Alberta; EXTRA Guest Faculty (Module 4: 2005–6) Stéphanie Rivard

consultant

Jean Rochon expert associé, Systèmes de soins et services, National Public Health Institute of Quebec, INSPQ, Montreal, Quebec; EXTRA Advisory Council Member and Guest Faculty (Module 4: 2005–8) Lorna Romilly

president, Romilly Enterprises

Denis A. Roy directeur, Gestion de l’information et des connaissances, Agence de la santé et des services sociaux de la Montérégie, Longueuil, Québec Andrea Seymour assistant deputy minister, Internal Services Alignment Project, Supply and Services, Government of New Brunswick, Fredericton, New Brunswick; EXTRA Graduate 2007, Intervention Project Title: Advancing Employee Engagement through the Implementation of a Healthy Workplace Strategy Samuel B. Sheps director, Western Regional Training Centre for Health Services Research Faculty of Medicine; professor, School of Population and Public Health, University of British Columbia, Vancouver, British Columbia; EXTRA Guest Faculty (Module 1: 2006–8)

204

Lyn Shulha

Contributors

Queen’s University, Kingston, Ontario

Ingrid Sketris professor, College of Pharmacy, Dalhousie University, Halifax, Nova Scotia Micheline Ste-Marie associate director, Professional Services, McGill University Health Centre, Montreal, Quebec; EXTRA Graduate 2009 Nina Stipich director, Executive Training, Canadian Health Services Research Foundation, Ottawa, Ontario David Streiner professor emeritus, Clinical Epidemiology and Biostatistics, joint member, Department of Psychiatry and Behavioural Neurosciences, professor, Department of Psychiatry, University of Toronto; director, Kunin-Lunenfeld Applied Research Unit, assistant vice president, Research, Baycrest Centre for Geriatric Care, Toronto, Ontario; EXTRA Lead Faculty (Module 2: 2004–8) Carl Taillon directeur, Direction des services professionnels, Centre hospitalier universitaire de Québec, Ste-Foy, Quebec; EXTRA Graduate 2007, Intervention Project Title: Efforts towards Medical Mobilization in a University Medical Center Richard Thornley coordinator, Impact Analysis, Alberta Heritage Foundation for Medical Research, Edmonton, Alberta Stphen Tomblin professor, Department of Political Science, Memorial University of Newfoundland, St John’s, Newfoundland Muriah Umoquit research associate, Cancer Services, Policy and Research Unit, Cancer Care Ontario, Toronto, Ontario; EXTRA Coordinator (Module 3: 2009–11)