Measuring Effectiveness in Humanitarian and Development Aid [1 ed.] 9781607419228, 9781600219597

Citation preview

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved. Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved. Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

MEASURING EFFECTIVENESS IN HUMANITARIAN AND DEVELOPMENT AID: CONCEPTUAL FRAMEWORKS, PRINCIPLES AND PRACTICE

No part of this digital document may be reproduced, stored in a retrieval system or transmitted in any form or by any means. The publisher has taken reasonable care in the preparation of this digital document, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained herein. This digital document is sold with the clear understanding that the publisher is not engaged in rendering legal, medical or any other professional services.

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved. Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

MEASURING EFFECTIVENESS IN HUMANITARIAN AND DEVELOPMENT AID: CONCEPTUAL FRAMEWORKS, PRINCIPLES AND PRACTICE

ANDRE M. N. RENZAHO EDITOR

Nova Science Publishers, Inc. New York

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2007 by Nova Science Publishers, Inc. All rights reserved. No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical photocopying, recording or otherwise without the written permission of the Publisher. For permission to use material from this book please contact us: Telephone 631-231-7269; Fax 631-231-8175 Web Site: http://www.novapublishers.com NOTICE TO THE READER The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained in this book. The Publisher shall not be liable for any special, consequential, or exemplary damages resulting, in whole or in part, from the readers’ use of, or reliance upon, this material. Independent verification should be sought for any data, advice or recommendations contained in this book. In addition, no responsibility is assumed by the publisher for any injury and/or damage to persons or property arising from any methods, products, instructions, ideas or otherwise contained in this publication.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

This publication is designed to provide accurate and authoritative information with regard to the subject matter cover herein. It is sold with the clear understanding that the Publisher is not engaged in rendering legal or any other professional services. If legal, medical or any other expert assistance is required, the services of a competent person should be sought. FROM A DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS. Library of Congress Cataloging-in-Publication Data Measuring effectiveness in humanitarian and development aid : conceptual frameworks, principles and practice / Andre M.N. Renzaho (editor). p. cm. Includes bibliographical references and index. ISBN-13: 978-1-60741-922-8 (E-Book) 1. Humanitarian assistance--Evaluation. 2. Economic assistance--Evaluation. 3. International relief--Evaluation. 4. Medical assistance--International cooperation--Evaluation. I. Renzaho, Andre M. N. HV553.M417 2007 331.2'6068--dc22 2007032049

Published by Nova Science Publishers, Inc.

New York

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

For…

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Catherine Renzaho, a loving wife, mentor, and an irreplaceable and inspirational teacher

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved. Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

CONTENTS About the Book Editor

ix

Contributors

xi

Foreword

xv

Preface Section I

Theoretical and Methodological Considerations

1

Chapter 1

Measuring Effectiveness in Development and Humanitarian Assistance: An Overview Andre M. N. Renzaho

3

Effectiveness: A Contribution from Australian Non-Government Organizations Linda Kelly

41

Defining and Refining Effectiveness: Applying Narrative and Dialogue Methods in Aid Monitoring and Evaluation Juliet Willetts, Helen Cheney and Paul Crawford

51

Chapter 2

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

xvii

Chapter 3

Chapter 4

Modelling the Impacts and Effectiveness of Aid Brett Parris

69

Chapter 5

Measuring the Effectiveness of HIV Prevention Bruce Parnell

93

Section II

Principles and Practice: Case Studies

115

Chapter 6

What Journalists Want: 'Selling' Humanitarian Emergencies to the Media Tim Large

117

Chapter 7

The Challenge of Attribution in Evaluating HIV Prevention Programs: Linking Inputs to Outcomes within a Multisectoral Response Mike Toole

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

139

viii Chapter 8

Chapter 9

Chapter 10

Reviewing the Evidence for Primary Health Care Effectiveness - Critical to Improving Local Health Systems and Community Health Chris Morgan and Michelle Kermode Raising the Funds – Spending the Funds: A Case Study of the Effectiveness of Both Roles of NGOs Matthew Clarke Effectiveness of Food Aid Programs in Response to the Food Crises in Southern Africa: Achievements, Challenges and Lessons Learned from Lesotho André M. N. Renzaho

155

173

187

Understanding the HIV and AIDS-Food Insecurity Vicious Cycle: Implication for Policy and Program Planning Danielle Pedi and Andre M. N. Renzaho

207

Good Practice in Microfinance: The Challenges of a Poverty Focus in an Evolving Industry Jamie Bedson and Andre M. N. Renzaho

233

Section III

Challenges

257

Chapter 13

Ethical Considerations in Humanitarian and Development Aid: What have We Learnt Over the Last 20 Years? Bebe Loff, Brad Crammond, Robyna Khan, Aasim Ahmad and Deborah Zion

Chapter 11

Chapter 12

Chapter 14 Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Andre M.N. Renzaho

Unpacking the Brain Drain in Sub-Sahara Africa through Public Health Lenses: Implications for Development Aid Andre M. N. Renzaho

259

279

Chapter 15

Toward Achieving Cultural Competence in Development Aid: Exploring Challenges for Public Health Practitioners and Putting a Framework Forward 301 Andre M. N. Renzaho

Section IV

Way Forward and Conclusion

323

Chapter 16

Shifting the Tipping Point on Achieving Realistic Outcome Parameters for Humanitarian and Development Aid Evelyne de Leeuw and Andrew McNess

325

Acknowledgement

339

Index

341

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

ABOUT THE BOOK EDITOR Dr Renzaho is a qualified and experienced Public Health specialist with highly developed leadership and management expertise with proven accomplishments in the area of development and humanitarian aid. He is currently a Senior Research Fellow within the School of Health and Social Development, Faculty of Health, Medicine, Nursing & Behavioural Sciences at Deakin University and the founder and director of the International Centre for Refugee Public Health and Nutrition. Over the last 17 years Dr Renzaho has been involved in various development and humanitarian aid initiatives at different levels including contractual agreement negotiations, government and multilateral agency representation, policy planning, analysis and development, advocacy, complex humanitarian emergency programming, UN, Government and NGO Relations, curriculum development and training delivery, program design, assessment and evaluation including Ex-post evaluations and establishing community partnerships and networks using participatory appraisal techniques. He is internationally renowned for his work in Public Health in complex humanitarian emergencies and among migrants from developing to developed countries. He has travelled extensively covering Australia, the Netherlands, France, Belgium, Singapore, Indonesia, Thailand, Ecuador, Lesotho, Tanzania, Mozambique, the Democratic Republic of Congo, Rwanda, Uganda, Kenya, Vanuatu, Laos, Malawi, and South Africa.

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved. Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

CONTRIBUTORS Professor Aasim Ahmad is a Chief nephrologist at the Kidney Centre and Honorary Senior Lecturer (Bioethics) at Aga Kahn University, Karachi, Pakistan. He is the chairperson of the Bioethics Group at Aga Khan University and a member of the National bioethics committee, Working Group on Disaster Research and Ethics (WGDRE) and Ethics Review Committees of Aga Khan University, Health Oriented Preventive Education (HOPE) and Médecins Sans Frontières (MSF). Mr Jamie Bedson Jamie Bedson has a BA in International Relations and a Master of Social Science in International Development. Past research and professional work has focused on community development issues including refugee resettlement, microfinance and water and sanitation. Jamie has experience with international development projects in Bosnia, Albania and Zimbabwe. He currently works as a private consultant and is assisting the Foundation for Development Cooperation on microfinance issues in the Asia-Pacific Ms. Helen Cheney has worked as a researcher, educator and manager in community, academic and government organisations, including roles at the Centre for Appropriate Technology, the Centre for Aboriginal Studies, Curtin University of Technology and the Institute for Sustainable Futures, University of Technology, Sydney. Helen is currently a Senior Research Fellow at the Australian Institute of Family Studies. Her fields of expertise are in community development and participatory evaluation. Dr. Matthew Clarke is a Senior Lecturer at Deakin University, Australia. Dr Clarke has co-written and co-edited a number of books and also published over 30 book chapters and refereed journal articles. Dr Clarke undertakes regular evaluations of community development projects in the Pacific and South-east Asia for various non-government organisations, with a particular interest in HIV/AIDS and health-related projects. He also researches human well-being, aid flows and climate change Mr. Brad Crammond has spent the last two years working on ways of applying human rights to health, particularly the practice of public health. His most recent project involved evaluating Public Health Association of Australia’s policy statements for their consistency with international human rights. Brad Crammond has further experience in examining the burden of health policies on human rights as part of a 5 year review funded by the NHMRC to determine the regulatory complement to factors influencing obesity. Brad Crammond is also currently writing his Master of Laws thesis on the interactions between health and human rights, focusing on the way human rights can be used as a basis for creating and assessing health policies.

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

xii

Andre M. N. Renzaho

Dr. Paul Crawford is a Monitoring and Evaluation (M&E) specialist with Aid-IT Solutions and an Adjunct Research Fellow at the Institute for Sustainable Futures where he completed his doctorate in M&E. Paul has consulted to a variety of bilateral, multilateral, NGO and contractor agencies throughout Asia, Africa, the Pacific, the Middle East, the Balkans and the former Soviet Union. He continues to be intrigued by how aid effectiveness can be enhanced. Professor Evelyne de Leeuw is Associate Dean (Development) and Chair of Health and Social Development with the Faculty of Health, Medicine, Nursing and Behavioural Sciences at Deakin University (with campuses in Melbourne, Geelong and Warrnambool in Victoria, Australia). She has set up or helped establish Schools of Public Health in countries such as Estonia, Denmark, El Salvador and Kazakhstan. Grounded in political science, she has been involved -since its inception- with the WHO Healthy Cities project around the world. Her current research is around dynamic multi-level policy network modelling. Her teaching is in the area of international health politics and policies, global health, and healthy cities. Dr. Linda Kelly is an independent consultant who has worked with Australian NGOs and with donor organizations across many fields, including monitoring and evaluation, activity design, community development and participatory development. She has ongoing engagement with the Australian Council for International Development. Dr. Michelle Kermode is a National Health and Medical Research Council Post-doctoral Public Health Fellow at the Nossal Institute for Global Health, the University of Melbourne. She is the principal investigator of the DFID-funded project “My First Time: Initiation into Injecting Drug Use in Manipur and Nagaland, Northeast India” and a co-investigator on a project seeking to Improve Mental Health as a Strategy for HIV Prevention among Vulnerable Women in India, also funded by DFID. Dr Robyna Khan is a physician who teaches at the Aga Khan University in Pakistan. She holds degrees in both Anaesthesia and in Bioethics. She is a member of the Aga Khan University hospital ethics committee and has been instrumental in the development and teaching of Bioethics in Pakistan. Mr. Tim Large has been AlertNet's deputy editor since 2003. Prior to that, he was a correspondent with Reuters in Tokyo, a staff writer on a major Japanese daily and news editor of a popular science website. He has written widely on politics, economics, social issues and the arts. He is also a passionate photographer. Associate Professor Bebe Loff, has worked for the Office of the High Commissioner for Human Rights (Geneva) with responsibility for HIV. She has examined the issue of adolescent consent to treatment for the World Health Organization and has provided advice in the area of human rights to the Joint United Nations Programme on AIDS. She has further provided informal advice to Professor Paul Hunt, the Special Rapporteur on the Right to Health in the areas of mental illness and intellectual disability. In addition Assoc/Prof Loff was, for some time, Manager of Policy and Legislation Review for the Health Department, Victoria, Australia, and prior to this a Policy Adviser in the Attorney-General’s Department. She co-ordinates the Masters of International Research Bioethics, a program funded by the NIH, at Monash University Dr. Andrew McNess is a sociologist with a background in research around coping with the loss of friends and relatives by young people. As a sociologist, he has a keen interest in people, and is an ambitious (amateur) actor and director.

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Contributors

xiii

Dr. Chris Morgan has been an International Health Specialist at the Burnet Institute Centre for International Health since 2001. He is also a Fellow in the School of Population Health of the University of Melbourne and Monash University. From 1993 to 1997 he worked in both hospital paediatrics and community health and development in western Nepal. From 1998 to 2000 he was team leader and primary health care advisor on the AusAID Tibet Primary Health Care and Water Supply Project. From 2001 to 2004, among other things, he worked as Child Health and Immunisation Advisor on the nation-wide Women’s and Children’s Health Project in Papua New Guinea. He has worked with government authorities on development of a Safe Motherhood Strategy for Tibet. Chris has also worked with WHO on various aspects of implementation of their Integrated Management of Childhood Illness program Mr Bruce Parnell is convenor of the Burnet Institute’s International HIV and Development Working Group. He is a public health specialist and philosopher with special interests in HIV and development, strategic planning, program design and evaluation. Bruce is a regular consultant to UNAIDS, UNDP, UNICEF and WHO. He recently reviewed ASEAN’s response to the HIV epidemic and developed the strategic framework for their next five years of responses. Bruce is based in Melbourne and works mostly in South East Asia. Dr. Brett Parris is a Senior Economic Adviser with World Vision Australia and a Research Fellow within the Dept. of Econometrics & Business Statistics at Monash University, Australia Ms. Danielle Pedi has a BA in Political Science/Community Development and a Master of Social Science in International Urban and Environmental Management. Her research and professional work focuses on social and policy aspects of natural resource management, in particular the interaction between human livelihoods and the natural systems on which they depend. Danielle has experience in environment and international development projects in the U.S., Australia and the Asia-Pacific. She currently works at the International WaterCentre in Brisbane, Australia. Assoc Prof Michael J. Toole, a medical epidemiologist and public health physician by training, has been the Head of the Centre for International Health of the Burnet Institute since 1995 and is also Associate Professor in the Department of Epidemiology and Preventive Medicine at Melbourne’s Monash University. From 1973 to 1994, he worked in rural and refugee public health programs in Thailand and Somalia, was the health coordinator of Oxfam/Community, and worked at the U.S. Centers for Disease Control and Prevention, where he coordinated the agency’s technical assistance to refugees and displaced populations in various countries. He is a board member of MSF Australia and a member of the Global Fund Technical Review Panel. Dr. Juliet Willetts is a Researcher Principal at the Institute for Sustainable Futures, University of Technology She specialises in transdisciplinary research that supports improvements to policy and practice in international development aid and sustainable water management. Her background includes a doctorate in Environmental Engineering from University of NSW, an extended period of in-country work in health education in India, and numerous research consultancies in Australia and internationally. Dr Deborah Zion has taught in the area of Ethics and Human Rights in undergraduate medicine, the Master of International Health and the Master of Public Health. She has also designed and implemented short courses on health ethics for healthcare professionals and ethics committee members. She has a special interest in the relationship between rights, ethics

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

xiv

Andre M. N. Renzaho

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

and public health, and in mental health. At present she is examining the rights based and ethical issues that arise in the context of health care practitioners who provide care and treatment for asylum seekers.

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

FOREWORD Almost forty years ago the Canadian Prime Minister and Nobel Peace Laureate Lester Pearson described the widening gap between developed and developing countries as the "central issue of our time." In response, Pearson called for the percentage of national income directed towards development to be raised to 0.7%, a target which to date has been reached by a tiny handful of nations, none of which are part of the G8. In fact, global development aid as a fraction of global GDP has fallen in the decades following Pearson’s appeal. This decline did not occur because more and more countries were reaching economic takeoff, and thus removing themselves as legitimate recipients for aid. Instead, in the decades following Person’s appeal, the gap which he described has become a gulf, which seems increasingly difficult to bridge. If the world were a single country then its distribution of income would now be far more unequal than within any single nation. This is true whether income is measured as a tradeable currency such as the Euro, or whether “international dollars” are used. These attempt to account for differences in purchasing power. It similarly follows that the distribution of political power, living standards, consumption patterns and health are also appallingly unfair. Relatedly, this also means that the gulf in power, perception and aspiration between donor and donee is enormous. Development aid has fallen for many reasons. One is the global shift towards a marketbased economy and society. This philosophy has encouraged the pursuit of private goods and social selfishness, rather than the raising of public and social goods, such as health, equity, sustainable ecosystems, and genuine freedom. The decline in foreign aid which has occurred as a partial consequence is unlikely to have been a primary goal of those who promoted market solutions. Apart from anything else the quantity of money devoted to aid, as a percentage of total spending, is too small to be a priority. One result of the pursuit of materialism and selfishness encouraged by market fundamentalism has been a loss of interest in international equity, sometimes euphemised as “donor fatigue”. It is unlikely that those who most celebrate and promote market forces would regret the decline in aid which has occurred, because proponents claim that market forces are the best way to facilitate development, health and governance in poor countries. Like many absurd claims, this is partly true, but free markets, without extensive checks and balances, are more likely to lead to catastrophe and despotism. The hope and achievements of the early post WWII years were stimulated and fuelled by decolonisation and the “low hanging fruit” which the newly emerging health technologies and institutions made possible. The newly formed World Health Organisation was one among several social, technological and economic forces which facilitated a rapid increase in life

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

xvi

Andre M. N. Renzaho

expectancy in many developing countries. These human forces were assisted by vaccines, DDT and first generation antibiotics and anti-malarials. But this period of activism and optimism has in many countries faded. Instead this has been exchanged for many devastating and interlinked problems. Among these include HIV/AIDS, poor terms of trade, exhausted soils, conflict, low education, high birth rates and anti-malarial drug resistance. However, the decline in the quantity of global aid has explanations which are far richer than the resurgence of global marketism. An important cause is disenchantment with the capacity of aid to genuinely reduce poverty. In part the explanation for apparent failure of this lies in the complexity of problems which need to be addressed and the policies, such as unfair trade, which undermine development. The quantity of global aid remains abysmally low. But critics also point out that aid depends critically on its quality as well as its quantity. Lobbyists have pointed to the growing phenomenon of “phantom aid” – funds claimed as aid in order to come closer to the Pearson target of 0.7%. Such claims have extended to include Australia, with recent allegations that up to one third of the Australian government’s recent aid budget is at best ineffective and at worst illusory. Other critics are concerned that the increasing quantity of untied aid from China will do little to promote sustainable development. In this context it is thus very important that the quality of aid can be better assessed. Such assessments, if acted upon, will not only improve the value of the aid dollar, but should also increase the total budget provided for aid, because evaluation will help aid providers to more effectively answer their critics. This book is a very worthwhile instrument with which to improve the quality of aid. To date, methods to assess the value of aid have been very scarce. In particular, the pendulum has swung too far in favour of satisfying the accountability of NGOs to their donors rather than to those they seek to help. This book brings together development practitioners, programme managers and academics to look at the issue. The collaboration of so many different and potentially competing stakeholders is itself a major achievement. It is to be hoped that this book will help to improve the quality of aid, the quantity of aid, transparency and human rights. Together these factors may then combine to improve the sustainable well-being of our fellow humans. If we can do that then our own sustainable wellbeing will also be protected and enhanced.

Dr. Colin Butler B.Med, DTM&H, PhD Medical Director Benevolent Organisation for Development, Health and Insight www.bodhi.net.au

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

PREFACE Ensuring effective intervention and resource utilisation is a stated goal of aid organisations, as well as an expectation of donors and the general public. Failure to be effective engenders criticism and loss of faith in the organization from donors, staff, beneficiaries and the general public alike. Lack of effectiveness in humanitarian and development aid also has a moral and ethical dimension which sets it apart from other types of work. Lack of effectiveness generates a pervasive cynicism that can be difficult to rectify. Hence the demonstration of effectiveness is central to humanitarian and development aid intervention in all its forms, and asks the question: How can we make humanitarian and development aid most effective? What do we mean by effectiveness? How can it be measured and by whom? Measuring the effectiveness of development aid and humanitarian programs remains a difficult task. In the development field in particular there is ongoing debate about how to measure effectiveness, and much development work still lacks assessment or evidence that it is having the intended positive impact, and avoiding negative consequences. With the proliferation of non-government organizations (NGOs), the NGO sector is made up of entities operating at different levels and with different capacities. The aggressive TV fundraising advertisements of some of the biggest NGOs in a bid to increase their revenue is a sad reality of the other side of NGOs’ endeavours. While no NGO would exist without fundraising, and hence the motivation for raising funds, the problem lies with the way fundraising is done, especially when funding-raising objectives take precedence over that of saving lives, improving lifestyles, promoting democracy and civic participation. It is a pity that fundraising indicators and financial growth are increasingly becoming the primary key performance indicators within the NGO sector, regardless of whether raised funds have been put to effective use. Some NGOs are becoming more media stars than development agencies. Raising money and good development are not just a case of throwing more money at an issue. As we discuss in chapter one, prioritizing fund raising over development objectives means that the NGO sector has become very competitive and NGOs have adopted business-like corporate models of governance, which to some extent, impact negatively on NGOs’ quest to achieve effectiveness within their programs. The primary objective becomes that of organisational survival, maintaining a field presence at any cost and financial growth over that of making a difference among those who need their services most.

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Andre M. N. Renzaho

xviii

This loss of focus on the humanitarian aspect of aid and development can be best seen when looking at how NGOs have taken up the opportunity to learn from past experiences. While there have been some successful stories such as the Sphere project (Humanitarian Charter and Minimum Standards in Disaster Response), the local capacity for peace (Do no Harm Framework) and the community-based therapeutic care, overall the last three decades has been dominated by over-documentation of mistakes and failure to learn from earlier projects. The first comprehensive documentation of mistakes started with the Joint Evaluation of Emergency assistance to Rwanda in 1996 (see chapter 1 for more details). Most of the pertinent lessons that were learnt in response to the Rwandan emergencies continue to occur and/or to be overlooked, and were recently documented in the tsunami response, inter alia (Schulze, 2005, Aguswandi 2005, Aspinal 2005): • • • •

Poor coordination and Interagency competition among international and local NGOs and the military Program duplication and overlap Culturally incompetent and inappropriate aid programs Raising unrealistic expectations among displaced people in the reconstruction efforts with little consideration to housing and land right issues and laws

In her report, “Between conflicts and peace: Tsunami aid and reconstruction in Aceh” Schulze notes that “the US agencies did not coordinate with the UN in the early days as there was a perception within the US about how useless the UN is… On occasion there was outright hostility between the US and UN agencies at coordination meetings” (2005:8). The author stresses that the competition between key players had four dimensions:

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

•

•

•

•

International NGO-local army relations e.g. International NGOs sceptical of military involvement in the response while the militaries saw international NGOs as spies, ineffective and uncoordinated do-gooders, Competition among international NGOs e.g. competition for projects, land/territorialism, houses and infrastructure which had not been destroyed, and local staff raising salaries and the cost of life in the process, International NGO-local NGO relations e.g. deliberate exclusion of local NGOs in operations and coordination, especially those with no international NGO partner, clashes in cultures, beliefs and values Geography e.g. International NGOs congregating in accessible areas where they had ready access to the media and could promote their projects

In describing the NGO business in Aceh, Schulze explains: There were two zones: the Banda Aceh and the West Coast zone. Banda Aceh is where all the lights and cameras were. Up there were all the heavies of the NGOs and the turf battles. Some NGOs were tearing down other NGO’s stuff. There was so much money they did not know what to do with it. There was no coordination of objectives. There were disparities among communities-one gets everything it wants, the other nothing. In the West Coast zone, in Meulaboh there were the real field officers and real Red Cross people-not guys with ties! They came and worked hard.

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Preface

xix

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

There were 54 INGOs in west Aceh. There were some disputes and much arbitration. OCHA was coordinating and generally it worked. The absence of lights and cameras helped (2005:12). It is these types of situations that make NGOs vulnerable to unnecessary criticism about their effectiveness and questioning about their motives. However, some NGOs have been showing the way forward in terms of transparency and sticking to organisational values. Consider, for example, Médecins Sans Frontières (MSF) in relation to the tsunami response. Like other NGOs, donors were generous toward MSF in response to the Indian Ocean Tsunami that occurred on December 24, 2004. The Tsunami generated an extraordinary generosity worldwide and all NGOs were winners and effective fund-raisers. MSF led the way demonstrating effective management and implementation of their aid. Rather than fighting other NGOs over adopting villages or worse affected areas in order to spend the money it received for the sake of spending or just maintaining a presence as played by other NGOs (see Schulze 2006), MSF was determined to stick to its values - to strive to bring medical assistance to those who most need it wherever they were. In its report “One year after the Indian Ocean tsunami disaster” MSF indicates that it received 110 million euros. Eight months from the onset of the disaster (i.e. by the end of August 2005), MSF had spent 20.9 million euros on tsunami-related operations. MSF forecast that it would only have spent 25 million euros by the end of 2005, and that this was more than enough to meet the needs of its constituencies. This budget was achieved by sticking to what they do best rather than trying to expand their programs to embrace areas in which they did not have expertise in order to spend the money received. In alignment with the organisational ethos, MSF started to contact its donors offering either to return some of the money it had received to help Tsunami victims or to reallocate the funds to the "forgotten" emergencies such as the Democratic Republic of the Congo, Niger, Somalia and the Darfur region of Sudan, where thousands and millions of people continue to die each year from preventable causes. These ongoing crises do not command the same level of attention as the tsunami did. Most donors were overwhelmingly positive in response to MSF’s offer to return funds; only 1% asked that their money be refunded and the majority allowed MSF to use the funds elsewhere. That is a true example of active advocacy by MSF and this is reflected in MSF activity report for 2005, when the organisation notes: Thanks to its supporters, MSF can retain its independence and intervene when and where our assistance is most needed. This impartiality and independence of analysis and action are crucial elements of MSF's identity that must be upheld despite the changing landscape of humanitarian aid

There have also been a number of other NGOs showing leadership in representing the voice of the deprived. For example, Oxfam has shown leadership in the area of advocacy, partnering the voiceless to make their voices heard. Oxfam has been one of the most vocal NGOs in exposing the suffering of Congolese, terming the war in the Democratic Republic of Congo the ‘Third World War’. Such level of representation was complemented by a series of mortality surveys by the International Rescue Committee in collaboration with the Burnet Institute, suggesting that nearly four million people have died as a result of the ongoing conflict (Coghlan et al 2006). While discussion of individual NGOs and the effectiveness of

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Andre M. N. Renzaho

xx

their programs is beyond this book, some of the chapters document successful stories and NGOs’ contribution to development as well as discussing some of gaps and dilemmas. As such, this book is the first to bring together development practitioners’ and academics’ perspectives on some of the issues described above, from fundraising strategies to effective program implementation. There is light at the end of the tunnel though. In Australia, World Vision Australia has spearheaded and funded a series of effectiveness conferences, the first of which informed the content of this book. In some corner of the world such as in the United Kingdom, fundraising coordination bodies such as the Disaster Emergency Committee (representing 13 leading UK aid agencies) are being formed as umbrella organisations to achieve two objectives: 1)

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

2)

ensure that NGOS do not prioritise fundraising and media attention over ensuring that aid is used effectively. The emphasis is on ensuring an effective, timely and fully accountable use of raised funds. To ensure that the known factors and/or components that render aid ineffective are not repeated. This means that maximising the learning from mistakes made in the past to improve standards in the implementation of humanitarian responses.

As such, this book is the first to bring together development practitioners and academics to look at some the issues described above, from fundraising to effective program implementation. The book is structured in four sections. Section 1 looks at the issues related to theoretical and methodological considerations. Chapter 1 to 5 deal with gaps in measuring effectiveness of development aid programs including issues related to evaluability assessment, outcome mapping and sustainability (Chapter 1), perspective in measuring effectiveness (Chapter 2), the application of narrative and dialogue methods in defining and redefining effectiveness (Chapter 3), the role of economic modelling in measuring the impact and effectiveness of aid programs (Chapter 4) and the outline of how the effectiveness of HIV prevention is currently being measured at a global level, and the requirements at local and national levels (Chapter 5) Section 2 looks at principles and practice with some case studies. Chapter 6, selling humanitarian emergencies to the media, focuses on the relationship between journalists and aid workers and suggests practical strategies relief agencies can take to raise the media profile of neglected emergencies. This is followed by a discussion on the challenges of attribution in HIV prevention programs, using Laos as a case study (Chapter 7). The chapter also discusses challenges in evaluating behaviour change and moves on to summarise knowledge related to outcome mapping approaches in HIV and AIDS programming. Given the large number of primary health care related projects in development aid and humanitarian programs, Chapter 8 summarises what works and what doesn’t. The chapter documents some successful Primary health care programs and highlights the important contribution of primary health care to the health and well-being of disadvantaged groups at the same time identifying deficiencies in practice and suggestions that need to be taken into account for future evaluations. The issue of whether effective fund raising translates into effective programming is extensively discussed in Chapter 9; drawing from the tsunami experience and this is complemented by an analysis of the effectiveness of food aid programs, using Lesotho as a case study (Chapter 10). Chapter 11 examines the relationship between HIV and AIDS and food insecurity through poverty lenses at the micro, meso and macro level, followed by a

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Preface

xxi

discussion on challenges of poverty focus in microfinance and micro-enterprise development programs (Chapter 12). Section 3 looks at dilemmas and challenges in measuring effectiveness, starting with the ethical considerations in development aid and humanitarian programs (Chapter 13), followed by a discussion of the challenges associated with the brain drain and its impact on achieving the Millennium Development Goals in sub-Saharan Africa (Chapter 14) before concluding the section with an analysis of the importance of cultural competence in public health research and practice for development aid programs (Chapter15). The last section proposes a way forward. In Chapter 16 we stand back from the current confusion and conflicts that dominate the discussion on measuring effectiveness of development aid and humanitarian program and take a broad view of possibilities for the future. The chapter describes six models to explain the interface between research, policy and practice, a framework that should form the future for measuring effectiveness in development aid and humanitarian assistance.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

REFERENCES Aceh rebuilding too clumsy, uncoordinated. The Jakarta Post 18 October 2005. Retrievable at Coghlan et al (2006). Mortality in the Democratic Republic of Congo: a nationwide survey. Lancet, 367(9504):7-9. Aspinal E. Paranoia and politics in the disaster zone. Sydney morning Herald. 3 February 2005 Médecins Sans Frontières (2005). One year after the Indian Ocean tsunami disaster. Retrievable at < http://www.msf.org/msfinternational/invoke.cfm?objectid=4779BD1E- 032328C7-D4BF508ECF63E0E6&component=toolkit.article&method=full_html>. Schulze, K (2005). Between conflicts and peace: Tsunami aid and reconstruction in Aceh. London: London School of Economics and Political Science

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved. Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

SECTION I

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

THEORETICAL AND METHODOLOGICAL CONSIDERATIONS

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved. Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

In: Measuring Effectiveness in Humanitarian… Editor: Andre M. N. Renzaho, pp. 3-40

ISBN 978-1-60021-959-7 © 2007 Nova Science Publishers, Inc.

Chapter 1

MEASURING EFFECTIVENESS IN DEVELOPMENT AND HUMANITARIAN ASSISTANCE: AN OVERVIEW Andre M. N. Renzaho

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

ABSTRACT Measuring effectiveness in humanitarian and development aid is an area that is poorly debated and understood. This chapter attempts to discuss challenges associated with measuring the effectiveness of development aid and humanitarian programs. It starts by examining gaps in measuring effectiveness, contrasting humanitarian assistance with development aid programs. The discussion starts by providing an historical perspective on the methodological gap, then moves on to address challenges related to determining the evaluation perspectives and outcome measures, evaluability assessment, and sustainability. The chapter concludes by advocating the need for capacity building in evaluation within development and humanitarian aid agencies. It suggests that this should be linked into a better understanding of the communities NGOs seek to serve through the use of a vulnerability assessment framework.

INTRODUCTION Measuring effectiveness in humanitarian and development aid is an area that is poorly debated and understood. Because of the varying degree of complexity, nature and political importance attached to international aid, it is important to delineate what is meant by ‘development’ and ‘humanitarian’ assistance before embarking on talking about measuring effectiveness in these types of programs. For the purpose of this chapter, development aid (also known as development assistance) refers strictly to any form of tied or untied economic assistance in the form of loans or grants by a developed country to a developing country either directly or through a multilateral agency or a non-government organization (NGO) (Reddy and Minoiu, 2006), and/or direct sponsorship programs funded by private donors to stimulate political change and to promote economic and social development. Government

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

4

Andre M. N. Renzaho

funding agencies include the U.S. Agency for International Development, the Australian Agency for International Development, the Canadian International Development Agency, the New Zealand’s International Aid and Development Agency, the Japan International Cooperation Agency, the UK Department for International Development just to name a few. These agencies have something in common, which is supposedly to support sustainable development and reduce poverty in line with their national interests through global health, agriculture and trade, economic growth, and democracy and conflict prevention. The above-mentioned agencies also provide humanitarian assistance. Humanitarian assistance refers to international transfers in some form of donations to assist a country experiencing the consequence of man-made and natural disasters. This will encompass both complex humanitarian emergencies and protracted and recovery operations. Emergencies will tend to be categorized based on the intensity and duration of the catastrophe. To this end, a complex humanitarian emergency has been defined as “... a relatively acute situation affecting a large civilian population, usually involving a combination of war or civil strife, food shortages, and population displacement, resulting in significant excess mortality”(Burkholder and Toole, 1995:1012). Chronic emergencies on the other hand include such things as the World Food Program’s Protracted Relief and Recovery Operations. Protracted and recovery operations are preferred in case of crop failure due to natural disasters (e.g. the Southern Africa Food Crisis) or when trying to sustain disaster-hit communities in order to help them re-establish their livelihoods and stabilize food security levels(World Food Programme, 2005). The assumption is that protracted crisis interrupts food production and significantly damages the foundations of people's livelihoods, thus eroding the social fabric of the affected communities. The objective of such programs is to restore the destroyed livelihood and the infrastructure disaster-hit people rely on to survive. These programs include Food for Education and Training programs, Food-for- Recovery such as Food-For-Work, relief programs for refugees mainly refugees and/or internally displaced people living outside their area of origin for a long time due to fear of persecution without achieving self-sufficiency; and extended relief programs for returning refugees and/or internally displaced people after a long period of absence, as a kick-start to their new lives (Borton, 1998; World Food Programme, 1991) In this chapter, we therefore examine some gaps in measuring effectiveness in humanitarian assistance and development aid, explore the challenges associated with measuring effectiveness and suggest a way forward.

GAPS IN MEASURING EFFECTIVENESS: HUMANITARIAN ASSISTANCE VERSUS DEVELOPMENT AID PROGRAMS A Historical Perspective on the Methodological Gap As a result of the increase in the number of man-made and natural disasters over the last four decades, there has been a proliferation of NGOs to meet the raised needs. However, many of the emerging small NGOs lack the knowledge and expertise required for an effective development aid and emergency response. This trend has combined with the competitions between NGOs for services, differing priorities, and varying expertise and access to source of

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Measuring Effectiveness in Development and Humanitarian Assistance

5

funding to make the coordination of humanitarian and development aid a difficult task requiring balancing between needs-based objectives with political objectives, which are often incompatible (Renzaho, 2006a). To address these discrepancies, various guidelines have been put forward to guide practices in order to improve effectiveness. Over the last two decades relief-oriented programs have come up with critical cut-off points to classify emergencies (Table 1-2) and standardized evaluation methodologies and cut-off points for decision making and to assess program effectiveness (Table 3). Such standardized methodologies and cut-off points have allowed emergency programs to have objective exit strategies. However, development aid programs have not enjoyed the same level of success in relation to measuring effectiveness. The last 25 years has seen development aid programs embroiled in re-defining a myriad of evaluation paradigms, re-inventing the wheel under different terminologies, with little guiding principles and added value as far as measuring effectiveness of developmental programs is concerned. Despite program evaluation having such a long history in the development field, methodological anarchy, lack of standardized methodologies and the use of indicators that are ambitious and difficult to measure continue to be the common threads. The development of standardized methodologies and cut-off points to assess the effectiveness of humanitarian assistance in emergencies has a complex history and went through different phases. For example, in response to the genocide that occurred in Rwanda in 1994, hundreds of NGOs poured into Kigali, the capital of Rwanda and others crossed the borders into Goma, in Zaire (now Democratic Republic of Congo) to assist Rwandan refugees. At the time when I was working with UNICEF implementing immunization programs, it came to light that some of these NGOs were opportunistic and some of them came to existence just to respond to the genocide-triggered Rwandan emergency. The number of NGOs responding was staggering, with more than 200 national and international NGOs involved in the emergency, the majority of which (nearly 100 NGOs) stationed in Goma town alone at the peak of the operations to assist refugees who crossed the border. Approximately 180 NGOs operated in Rwanda by late 1994 (Joint Evaluation of Emergency Assistance to Rwanda, 1996). Such a presence of NGOs presented a dilemma from an effectiveness point of view in terms of planning, coordination, information sharing and competition. There were virtually no technical guidelines for planning and implementing programs apart from the 1978 World Health Organization first technical guidelines on the management of nutritional emergencies in large populations (Ville de Goyet et al., 1978) and the 1983 first textbook on refugee community health care (Simmonds et al., 1983). In the absence of technical guidelines, NGOs responding to the 1994 Rwandan emergency were faced with significant duplication of efforts and inefficient use of resources (Joint Evaluation of Emergency Assistance to Rwanda, 1996). The competition for resources between NGOs not only inflated the price of locally-produced commodities, making life difficult for the local Congolese population which had no right or access to international assistance, but also made NGOs look selfish and distrustful. NGO relationships with both the host population and the refugee community became compromised and NGOs were often accused of turning a blind eye on military and political control of refugees and the manipulation of humanitarian assistance in the various refugee camps in the bid to stage an immediate return to power in the country of origin (Toole, 1997). As a consequence, the extent to which aid agencies could decide to stay or go in light of knowledge that some aid was being provided to the Hutu

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Andre M. N. Renzaho

6

extremists became impaired and NGOs struggled about balancing NGOs’ primary core value of saving lives with the protection of the affected population from human rights abuses. Table 1: Definitions and benchmarks for interpreting nutritional data Indicator Wasting *

Definition Weight-for-height less than -2 Z-score or 80% of the median in children aged 6-59 months (a)

Stunting *

Height-for-age less than -2 Zscore or 90% of the median in children aged 6-59 months (a)

Benchmark for guidance in interpretation 10-14 % ‘undoubtedly high and indicating a serious situation’ • >15 % very high ‘a severe crisis’ • •

• • • •

Underweight *

Weight-for-age less than -2 Zscore or 75% of the median in children aged 6-59 months (b)

• • •

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

•

Oedema **

Clinical sign of kwashiorkor

•

Crude mortality rate ***

The number of all deaths within a specific time period

• • • • •

Under-five mortality rate **

The number of deaths among children under five years of age within a specific time period

• •

=40% very high =30% very high ‘Any prevalence detected is cause for concern’ 0- 0.5/10,000/day: Normal rate for developing countries 0.6-0.9/10,000/day: Relief program: under control 1-2/10,000/day ‘serious situation’ 2-5/10,000/day ‘emergency out of control’ >5/10,000/day: Relief program: major catastrophe 2/10,000/day ‘serious situation’ 4/10,000/day ‘emergency out of control’

(a) % of the median using Waterlow’s cut-off points (Waterlow, 1972), (b) % of the median using Gomez’s cut-off points (Gomez et al., 1956); * (World Health Organisation, 1995); **(Howe and Devereux, 2004), and *** (Medecins Sans Frontieres, 1995)

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Measuring Effectiveness in Development and Humanitarian Assistance

7

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

The challenges that NGOs, both big and small, experienced during the response to the Rwandan emergency highlighted the need to professionalize the sector and to develop guiding principles for emergency response. In 1994, under the leadership of the Red Cross and Red Crescent Movement, six networks of NGOs met to draw up a professional Code of Conduct outlining universal basic standards which should govern emergency response. The meeting resulted in 10 self-policing, obligation free points of principles that NGOs should apply when involved in emergencies (Box 1 and Box 2). Today, more than 300 NGOs are signatories. The development of Code of Conduct was shortly followed by the publication of a standardized emergency response framework by Médecins Sans Frontières, identifying the 10 public health priorities in the early phase of emergencies and the required long-term health strategies once mortality and risk of infectious outbreaks are under control (Medecins Sans Frontieres, 1995; Medecins Sans Frontieres, 1997), the Sphere Humanitarian Charter and Minimum Standards in Disaster Response, launched in 1997 and updated in 2004 (The Sphere Project, 2004), and the “Do No Harm” framework by Mary Anderson (Anderson, 1999). However, being a signatory does not necessarily guarantee or translate into adherence to the Code of Conduct. Over the last decade cases of NGOs violating code of conducts at all levels have been documented, including the use of humanitarian assistance as a tool for sexual exploitation, competition between NGOs, funds being allocated based on geographic affinity and political demarcation rather than needs, conceptually separation of relief from development at organisation and field levels, and militarisation of aid and reconstruction (Carballo and Solby, 2001b; Gillan and Moszynski, 2002; Renzaho, 2006a; Susser, 1991; Susskind, 2003).

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

8

Andre M. N. Renzaho

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Table 2: Scale for classifying the intensity of famine

Source: (Howe and Devereux, 2004). CMR= Crude Mortality Rate

Measuring Effectiveness in Humanitarian and Development Aid, Nova Science Publishers, Incorporated, 2008. ProQuest Ebook Central,

Measuring Effectiveness in Development and Humanitarian Assistance

9

Table 3: Decision making and programming Malnutrition rate >=15% Or Malnutrition Rate 10-14% + Aggravating factors* Malnutrition rate 10-14% Or Malnutrition rate 5-9% + Aggravating factors*

Malnutrition rate