But application of a theorybased approach remains weak. Evaluation department 2004, ravallion 2001, 2005, white 2006, and the methodological guidelines and impact evaluation case studies on the banks poverty impact analysis, monitoring and evaluation website. The impact evaluation model can support this process by helping. Core elements of a welldesigned impact evaluation 320. Assessment is frequently confused and confounded with evaluation. Beginning with the need for a theorydriven evaluation, and ending with a set of concrete tools to use in running an evaluation, they cover a range of practical materials useful. This book offers an accessible introduction to the topic of impact evaluation and its practice in development. An evaluation which looks at the impact of an intervention on final welfare outcomes, rather than only at project outputs, or a process evaluation which focuses on implementation. The proper analysis of impact requires a counterfactual of what those outcomes would have been in the absence of the intervention. Quality in policy impact evaluation qpie is a supplement to the magenta book and provides a guide to the quality of impact evaluation designs. Tennessee department of education revised november 2018. The evaluation practice handbook offers comprehensive information and practical guidance on how to prepare for and conduct evaluations in who, and gives guidance. Alternatively, a program that is viewed as a failure due to budget and schedule issues may.
Calls for rigorous impact evaluation have been accompanied by the quest not just to find out what works but why. Goldilocks assembled this set of resources for use in designing and running an impact evaluation. The formal literature on impact evaluation methods and practices is large, with a few useful overviews. Improving the quality of data and impactevaluation. Impact evaluation has been defined differently over the past few decades. Introduction unleashing the potential of continuous improvement in teachinglearning requires an appreciation of the difference in spirit between assessment and evaluation. The logframe see module 1 provides the starting pointing for integrating impact assessment with. The monitoring and evaluation handbook for business environment reform. Evaluations that are being undertaken to support learning should be clear about who is intended to learn from it, how they will be engaged in the evaluation process to ensure it is seen as relevant and credible, and whether there are specific decision points. Quantitative and qualitative methods in impact evaluation and measuring results 1 1 introduction there has been a renewed interest in impact evaluation1 in recent years amongst development agencies and donors. This2009versionofthehandbookonplanning,monitoringandevaluatingfor developmentresultsaimstosupportundpinbecomingmoreresultsorientedand. Impact assessment focuses on the effects of the intervention, whereas evaluation is likely to cover a. Finding adequate sources of data 291 kinds of data that are needed 291 using existing quantitative data 294 collecting new survey data 299 chapter 17.
It has been developed to aid policy makers and analysts understand and make choices about the main impact evaluation designs by. Outline of principles of impact evaluation part i key concepts definition impact evaluation is an assessment of how the intervention being evaluated affects outcomes, whether these effects are intended or unintended. Goldilocks resource resources and tools for impact evaluation. Yet there is a need to put the theory into practice in a handson fashion for practitioners. Skills development impact evaluation a practical guide. Kirkpatricks four levels of training evaluation in detail this grid illustrates the kirkpatricks structure detail, and particularly the modernday interpretation of the kirkpatrick learning evaluation model, usage, implications, and examples of tools and methods. The difference in the outcome of interest between having or not having the program or intervention is known as its impact, and measuring this difference. The term implies a broad evaluation that considers unintended consequences. Impact evaluation of third elementary education project in the republic of the philippines pdf 1. Different impact evaluation methodologies differ in how they construct the counterfactual counterfactual. This approach to evaluation focuses on the practicalities of defining successful outcomes and success cases brinkerhoff, 2003 and uses some of the processes from theorydriven evaluation to determine the linkages, which may take the form of a logic model, an impact model, or a results map. Skills development impact evaluation introduction why is a skills development impact evaluation guide necessary.
Impact analysis for program evaluation, second edition has been revised to cover new issues and to further clarify the concepts used in impact analysis. This is a transparent, flexible and relatively simple set of guidelines for collecting, analysing and sharing feedback from intended beneficiaries about significant drivers of. Impact evaluation methodologies impact evaluation workshop, lilongwe, dec. The second edition of the impact evaluation in practice handbook is a comprehensive and accessible introduction to impact evaluation for policy makers and development practitioners. During the last decades, many impact evaluation experiences have been carried out by member institutions of the network coordinated by ilocinterfor with a view to finding practical solutions that suit their needs. Impact is a measure of the changes made and impact assessment seeks to establish a causal connection between inputs and changes in terms of magnitude or scale or both. First published in 2011, it has been used widely across the development and academic communities.
It is widely accepted that a theorybased approach to impact evaluation, one that maps out the causal chain from inputs to outcomes and impact and tests the underlying assumptions, will shed light on the why question. This additional material is made freely but please acknowledge its use as follows. Quantitative methods and practices makes a valu able contribution in this area by providing, for policy and research audiences, a com. Other interpretations of impact evaluation include. Impact evaluation methods experimental design presented by. The evaluation purpose refers to the rationale for conducting an impact evaluation. Examples of outcome and impact indicators are presented in figure 2. Broadly, evaluation aims to understand to what extent and how a policy intervention corrects the problem it was intended to address.
Yet there is a need to put the theory into practice in a handson fashion for. This site is like a library, you could find million book here by using search box in the header. This material constitutes supporting material for the impact evaluation in practice book. Quantitative and qualitative methods in impact evaluation. Quantitative methods and practices world bank training series khandker, shahidur r. Evaluation measures how well the program activities have met expected objectives andor the extent to which changes in outcomes can be attributed to the program or intervention. Overview of impact evaluation page 2 an impact evaluation should only be undertaken when its intended use can be clearly identified and when it is likely to be able to produce useful findings, taking into account the availability of resources and the timing of decisions about the programme or policy under investigation. For more on using a theory of change and logic model to inform an evaluation plan, see. Recognizing the importance of impact evaluation to unicefs work, the office of research innocenti has collaborated with rmit university, betterevaluation and the international initiative for impact evaluation 3ie to produce a series of methodological briefs and videos on the topic. This book has been made possible thanks to the generous support from the spanish.
This is part of the evaluation and management activity of the policy cycle. Attributing development impact brings together responses to this challenge using an innovative impact evaluation approach called the qualitative impact protocol quip. Kirkpatricks four levels of training evaluation in detail. Impact evaluation of infrastructure interventions pdf. Did the policy produce the intended outcomes and impacts. Impact evaluation a model guidance and practical examples. It offers expanded coverage and explanation of quasiexperiments, a new section on the theory of impact analysis, updated information on the use of qualitative research for impact analysis, and. Choosing an impact evaluation method 187 determining which method to use for a given program 187 how a programs rules of operation can help choose an impact evaluation method 188 a comparison of impact evaluation methods 193 finding the smallest feasible unit of intervention 197 chapter 12. All books are in clear copy here, and all files are secure so dont worry about it. Evaluating policy impact step by step evaluating violence and injury prevention policies policy impact evaluation. Impact analysis for program evaluation sage publications. Read online improving the quality of data and impactevaluation.
697 1285 323 467 537 487 207 581 1174 1041 227 1326 576 1215 197 1122 1093 179 1438 794 451 1330 1090 457 1362 809 1109 1324 1047 1208 1047 774 94 139 212 1254 1340 1015 289 383 786 1090 1007 729 429 488 519