Assessments, Monitoring and Evaluation

See Also: Food Aid; Food Security; HIV/AIDS; Household Food Consumption; Infant & Child Nutrition; Layers, A Computerized Monitoring Tool; Women & Adolescent Nutrition

USAID defines monitoring and evaluation in ADS 203 as:

"Performance management is the systematic process of monitoring the results of activities; collecting and analyzing performance information to track progress toward planned results; using performance information to influence program decision making and resource allocation; and communicating results achieved, or not attained, to advance organizational learning and tell the Agency's story."

Monitoring and evaluation (M&E) is a key area of FANTA's technical support to USAID and the development community. FANTA strengthens the M&E capacity of PVOs working on USAID-supported nutrition and food security activities through technical assistance committed to performance management and advancing organizational learning. Meeting USAID's performance management standards provides the framework for FANTA's assistance.

FANTA's technical assistance activities include workshops, publications, and in-country assistance, offered at all stages of the development cycle, from development activity design, developing the baseline data collection, to reviewing annual, mid-term, and final evaluation reports. Monitoring and evaluation skills, such as design, data collection, and data analysis, are core to organizations from both a performance and management standpoint. FANTA addresses USAID and PVO monitoring and evaluation needs with technical expertise and indepth understanding of both the M&E process and the nutrition and food security sectors.

		A brief description of each publication is provided below. By choosing the publication title, you are able to read a detailed description and to download the publication.

1. A Guide to Monitoring and Evaluation of Nutrition Assessment, Education and Counseling of People Living With HIV (2008): This publication provides guidance and tools to support programs in monitoring and evaluating nutrition interventions for people living with HIV (PLHIV). It is designed for use by program managers, M&E officers and other program and government health system staff who are responsible for designing and implementing M&E systems. The guide can be used to select indicators, set targets, plan data collection and tabulation processes and interpret and use the information obtained.

2. Precision, Time, and Cost: A Comparison of Three Sampling Designs in an Emergency Setting (2008) An article reporting on the FANTA and SC/US field test in Sudan, "Precision, Time, and Cost: A Comparison of Three Sampling Designs in an Emergency Setting", has been published in Emerging Themes in Epidemiology.

3. Trigger Indicators and Early Warning and Response Systems in Multi-Year Title II Assistance Programs (2007): USAID's Office of Food for Peace (FFP) and FANTA have published FFP Occasional Paper 5, Trigger Indicators and Early Warning and Response Systems in Multi-Year Title II Assistance Programs to allow Cooperating Sponsors (CSs) a greater degree of flexibility in responding to emerging crises and shocks in their areas of operation without the risk of potentially undermining advances being achieved by development interventions. Occasional Paper 5 is designed to briefly review CSs' experiences with operationalizing trigger indicators (TIs) and early warning and response (EWR) systems to-date, outline the key characteristics of EWR systems and TIs within the Title II-supported multi-year assistance program (MYAP) context, and provide suggestions on how to best operationalize FFP guidance on incorporating EWR mechanisms, including TIs, into MYAPs.

4. Alternative Sampling Designs for Emergency Settings (2007): Humanitarian agencies and decision makers need tools for rapid and effective prioritization of areas in greatest need and practical and reliable methods for on-going monitoring of the situation. There has long been demand for a statistically reliable emergency assessment tool that is less time and resource intensive than a conventional 30x30 cluster survey. Alternative Sampling Designs for Emergency Settings explores the potential of applying LQAS methods for rapid assessment of the prevalence of acute malnutrition.

5. Version 3 of Household Food Insecurity Access Scale (HFIAS) for Measurement of Food Access: Indicator Guide (2007): FANTA, in collaboration with Cornell and Tufts Universities, has developed a Household Food Insecurity Access Scale (HFIAS) measure and a guide, "Household Food Insecurity Access Scale (HFIAS) for Measurement of Food Access: Indicator Guide," with a standardized questionnaire and data collection and analysis instructions. The HFIAS is composed of a set of nine questions that have been used in several countries and appear to distinguish food insecure from food secure households across different cultural contexts. These questions represent universal domains of the experience of insecure access to food that can be used to assign households and populations along a continuum of severity. The information generated by the HFIAS can be used to assess the prevalence of household food insecurity (access component) (e.g., for geographic targeting) and to detect changes in the food insecurity situation of a population over time (e.g., for monitoring and evaluation). The questions can be added to a standard baseline and final evaluation survey. In August 2007, Version 3 of the guide was released. The new version offers an updated questionnaire section.

6. A Field Test of Three LQAS Designs to Assess the Prevalence of Acute Malnutrition (2007): A study by FANTA, Catholic Relief Services (CRS), and Ohio State University (OSU), published in the International Journal of Epidemiology, field-tested the use of the LQAS designs in an emergency setting in Ethiopia. The study "A Field Test of Three LQAS Designs to Assess the Prevalence of Acute Malnutrition," concluded that LQAS designs provide statistically appropriate alternatives to the more time-consuming 30 x 30 cluster survey though additional field testing is necessary.

7. Report of the CRS/Haiti DAP 2002-2007 Mid-term Evaluation (2006) [831 kb]

8. Household Dietary Diversity Score (HDDS) for Measurement of Household Food Access: Indicator Guide and Months of Adequate Household Food Provisioning (MAHFP) for Measurement of Household Food Access: Indicator Guide (2007): In light of the need to build consensus on household food access impact indicators, two strategic objective level indicators of household food access (HDDS) and months of inadequate household food provisioning (MIHFP) were identified during the development of USAID Office of Food for Peace (FFP)'s FY05-08 strategy, through a process of consultation and discussion with CSs, researchers, and other technical groups. These two indicators focus on the desired outcome of improved food access - improved household food consumption. FFP will be requiring all new Title II Multi-Year Assistance Programs (MYAP) with improved household food access as an objective to include these indicators in their results frameworks. In addition, a Household Food Insecurity Scale (HFIS) to measure the experience of household food insecurity is being tested for future inclusion as an indicator. These two indicator guides provide background on the indicator as well as guidance on data collection (including questionnaire format) and analysis.

9. Presentation on Implications of the New WHO 2006 Child Growth Standards, July 2006: In April 2006, the World Health Organization (WHO) released new growth standards for children 0-5 years, replacing the NCHS 1978 child growth references as the international standard. This presentation provides information on what the new WHO child growth standards are, why they were developed, and how the new WHO child growth standards may affect population-level data for program monitoring, evaluation, and decision making. The presentation was originally hosted by the Core Group and presented during an Elluminate Live online session on July 27, 2006.

10. Journal of Nutrition Supplement, May 2006: The Journal of Nutrition has been the principal forum for disseminating US-based research on food insecurity scales. It has also published most of the studies dealing with the application of food insecurity scales in developing countries. Publication of the process of developing the Household Food Insecurity Access Scale (HFIAS) offers the opportunity to continue this "conversation" by presenting, in one place, one of the most significant advances in food insecurity measurement to emerge over the last decade. The Journal of Nutrition Supplement, published in May 2006, presents the results of the FANTA-led HFIAS initiatives, including: findings of the first studies to develop and validate food insecurity scales from the "ground-up" in a developing country context; findings of the first cross-country study to compare the performance of the same set of food insecurity items in four dramatically different cultures; results of a cross-country review of food insecurity scale applications and its implications for the development of a universal measurement tool; and, status of the process to develop a universal measurement tool.

11. Evaluating Title II Development-oriented Multi-Year Assistance Projects (2006): Evaluating a Title II development-oriented multi-year assistance project (MYAP) involves assessing its outcomes and impacts, that is, verifying the extent to which project activities are associated with intended changes in the practices and well-being of the beneficiary population. Evaluation objectives may range from simply measuring the level of change in indicators of well-being, to attributing a change in the level of those indicators to the intervention being implemented. The focus of this Technical Note is to lay out the various evaluation design options open to Title II project managers.

12. Monitoring and Evaluation Framework for Title II Development-oriented Projects (2006): All Private Voluntary Organizations (PVOs) submitting a Title II development oriented Multi-Year Assistance program (MYAP) proposal to USAID’s Food for Peace (FFP) must include a Monitoring and Evaluation (M&E) plan as part of their submission. According to USAID guidelines, the aim of the M&E plan is to measure the extent to which the activity will result in changes in behavior and well-being at the population level, as well as progress in activity implementation. This Technical Note explains how to frame a Monitoring and Evaluation (M&E) system that fulfills those functions, while maximizing its usefulness to project managers.

13. Development and Validation of an Experience-based Tool to Directly Measure Household Food Insecurity Within and Across Seasons in Northern Burkina Faso (2004): FANTA funded two multi-year field validation studies that used the HFSS approach to develop experiential food insecurity scales and validate them primarily as impact indicators for the access component of household food security (see Measuring Food Insecurity: Going Beyond Indicators of Income and Anthropometry below). The objectives of the studies were to develop a household food access measure (Household Food Insecurity Scale: HFIS) based on locally recognized behaviors that distinguish food insecurity in developing countries, test the HFIS's relationship to conventional indicators of food insecurity (such as income or food consumption), and test the HFIS's performance and sensitivity to change related to program impact.

14. Local Capacity Building in Title II Food Security Projects: A Framework (2004): This paper establishes a conceptual framework for local capacity building within food security projects. It is designed to provide Title II policy-makers and cooperating sponsors with a basic reference tool for the design, implementation, monitoring and evaluation of projects’ capacity building activities at the local level. This framework builds on the USAID food security framework, in which food availability, access and utilization constitute the three pillars of food security.

15. Measuring Household Food Consumption: A Technical Guide (2004): The guide describes the process and procedures for collecting information to assess the food intake requirements of a household and a step-by-step analysis of the food consumed. Appendices present detailed information about analyzing the data.

16. Child Care, Health and Nutrition in the Central Plateau of Haiti: The Role of Community, Household and Caregiver Resources (2003): This report presents the main findings of a baseline survey carried out as part of an IFPRI-Cornell University evaluation of World Vision International’s integrated maternal and child health program in Haiti’s Central Plateau. The goal of the overall study is to compare the recuperative approach to food supplementation, which targets malnourished children identified through regular growth monitoring activities, with the preventive approach, which targets all children six to 23 months in an attempt to prevent malnutrition rather than intervene once malnutrition is already established.

17. Generating Indicators of Appropriate Feeding of Children 6 through 23 Months from the KPC 2000+ (2003): This report suggests small modifications to the Knowledge, Practice, and Coverage Survey (KPC) 2000+ Infant and Child Nutrition module and tabulation plan to improve the ability to measure, interpret, and analyze several key infant and young child feeding practices of children in the 6 through 23 month age range. The authors also make recommendations for presenting infant and young child feeding practices survey results.

18. Measuring Food Insecurity: Going Beyond Indicators of Income and Anthropometry (2003): FANTA funded two multi-year field validation studies that used the HFSS approach to develop experiential food insecurity scales and validate them primarily as impact indicators for the access component of household food security (see Development and Validation of an Experience-based Tool to Directly Measure Household Food Insecurity Within and Across Seasons in Northern Burkina Faso above). The objectives of the studies were to develop a household food access measure (Household Food Insecurity Scale: HFIS) based on locally recognized behaviors that distinguish food insecurity in developing countries, test the HFIS's relationship to conventional indicators of food insecurity (such as income or food consumption), and test the HFIS's performance and sensitivity to change related to program impact.

19. Report on the 2002 Joint Baseline Survey in the Targeted Areas of the PL480, Title II Programs in Haiti (2003): When beginning a new program, Title II CSs must conduct a baseline survey assessing the conditions in target areas before activities begin. This provides a basis against which the progress made by the DAPs towards fulfilling their objectives can be assessed at the end of the project. This report communicates the results from the baseline survey conducted jointly by four cooperating sponsors in their targeted areas between May and July 2002 as a prelude to their new 2002-2006 programs. The findings and recommendations we provide in this report are valid for these targeted areas and cannot be extrapolated to other areas.

20. Anthropometric Indicators Measurement Guide (2003): This guide, revised in 2003, focuses on the anthropometric assessment of infants and children to assist PVOs in improving their M&E. It includes information on how to collect, analyze, and report on key anthropometric indicators.

21. Food Access Indicator Review (2003): The measurement of food access is critical to food security programming. However, for most Title II Cooperating Sponsors, determining changes in food access has not been easy, particularly because appropriate indicators are not standardized and are hard to measure and interpret. Guidance and tools to assist in measuring access indicators are limited or not readily available to the field. The objective of this study was to review how Title II Development Assistance Programs designs address food access, assess how Title II PVOs currently monitors and evaluates food access and identify good measurement practices. The results of the review will provide the basis for a follow-on food access monitoring and evaluation guide to be used by CS field staff.

22. Dietary Diversity as a Household Food Security Indicator (2002): Both the report and the technical note describe a user-friendly, cost-effective approach to measuring changes in dietary quantity and quality and feeding behaviors at both the household and individual levels. Dietary diversity, defined as the number of unique foods consumed over a given period of time, appears to show promise as a means of measuring food security and monitoring changes, particularly when resources for such measurement are scarce. As described in the report, FANTA's subcontractor, the International Food Policy Research Institute (IFPRI), analyzed ten household data sets (collected in India, Mozambique, Mexico, Bangladesh, Egypt, Mali, Malawi, Ghana, Kenya and the Philippines) to assess whether dietary diversity can be used as a tool in evaluating the efficacy of food security interventions. The study validated the dietary diversity indicator as a measure of access to food (per capita expenditures) and a measure of consumption (caloric availability at the household level).

23. Title II Evaluation Scopes of Work (2002): This technical note is an excellent reference for managers, program implementers and evaluation teams as it provides standard items to include in Scopes of Work for process and impact evaluations. Examples of evaluation questions are given for several sectors: agriculture, infrastructure, microenterprise and microfinance, health and nutrition, and education. The evaluation questions are illustrative and can be tailored to the specific characteristics or emphases of each development assistance program.

24. Food for Education Indicators Measurement Guide (2001): USAID supports Food for Education (FFE) approaches to increase enrollment and school attendance in developing countries. FANTA's guide was developed with Title II food aid PVOs, various USAID offices involved in education, and the World Bank and World Food Programme.

25. Building Household Food Security Measurement Tools from the Ground Up (2001): Wolfe and Frongillo's paper explores the potential to develop improved measures of the access component of household food security. The report discusses relevant conceptual and measurement issues and reviews the U.S. approach and examples of efforts in developing countries. It includes an outline of the elements needed to apply this approach, along with operations research needed for developing such experiential-based measures.

26. Water and Sanitation Indicators Measurement Guide (1999): This guide provides information on water and sanitation impact and monitoring indicators. Water and sanitation improvements, in association with hygiene behavior change, can lead to reduced morbidity and mortality and improved nutritional status.

27. Food Security Indicators and Framework for Use in the Monitoring and Evaluation of Food Aid Programs (1999): Integrating food security indicators into the monitoring and evaluation systems of food aid programs will ensure better and more efficient management of these resources and improve their impact. This guide outlines a process for identifying indicators and provides a conceptual framework for understanding food security issues.

28. Sampling Guide (1997): Sample surveys are often the most feasible way of gathering the information required for Title II program evaluations. This guide provides guidance on how to choose samples of communities, households, and/or individuals for these surveys. This information will permit analysis of the effectiveness of Title II programs when combined with appropriate indicators and evaluation study designs. This guide emphasizes the use of probability sampling methods. The Sampling Guide is also available in French and Spanish.

29. Agricultural Productivity Indicator Measurement Guide (1997): This guide discusses the following indicators and the data needed for each of them: harvested crop yields per hectare, gap between actual and potential yields, yield variability under varying conditions, value of crop production per household, months of household food provision, percent of crop losses during storage, number of hectares with improved practices, and number of crop storage facilities built and used.

top

*Please note that links to these sites do not imply that FANTA supports either the organization listed or the views and content presented.

Africa Evaluation Association [http://www.afrea.org]
This is the website for the African Evaluation Association. The site provides a directory of African evaluators, country evaluation associations, a listing of training opportunities, and links to other resources.

CARE International Design, Monitoring and Evaluation (DME) [http://www.kcenter.com/care/dme/]
A website which contains CARE International documents on design, monitoring, and evaluation. Also contains many useful links to other M&E sites.

ELDIS, Development Information Gateway [http://www.eldis.org]
Very useful site from Eldis, a gateway to on-line information on development in countries of the South. Coverage includes social, economic, political and environmental issues. Major annual reports of relevant institutions can be accessed as well. The site has a simple search engine, but also features pre-prepared searches on a variety of topics (debt relief, agricultural marketing, etc.) and illustrative stories to put these topics into context.

ELDIS, HIV and AIDS Resource Centre [http://www.eldis.org/hivaids/index.htm]
Another site from Eldis that provides a more structured overview of the subjects than is possible through the search option. This link arranges organizational sources into useful groups, has a guide to development information both on-line and in print, and enables users to directly search the websites of national aid agencies.

Evaluation Cooperation Group (ECG) [http://www.ecgnet.org]
Evaluation Cooperation Group's (ECG) website, dedicated to foster collaboration and harmonization of evaluation work among the evaluation units of the Multilateral Development Banks.

Food and Agriculture Organization (FAO) [http://faostat.fao.org/default.aspx]
FAOSTAT is an on-line and multilingual database with records covering international statistics in production, trade, food balance sheets, food aid shipments, fertilizer and pesticides, land use and irrigation, forest products, fishery products, food quality control, population, and agricultural machinery.

ID21 Development Research Reporting Service [http://www.id21.org]
Funded by the UK Department for International Development (DFID), ID21 Development Research Reporting Service, database includes articles on evaluation.

InterAction International Evaluation Resources [http://www.interaction.org/evaluation/index.html]
InterAction has a working group on evaluation as well as resources that can be accessed from this page.

International Institute for Sustainable Development (IISD) [http://www.iisd.org/measure/]
The International Institute for Sustainable Development (IISD) is conducting a major update of the online Compendium of Sustainable Development Indicator Initiatives and Publications. Provides access to over 10,500 titles relating to sustainable development.

MandE News [http://www.mande.co.uk/]
MandE News provides information on M&E materials, people, events, and networks.

Population Reference Bureau [http://www.prb.org/]
The Population Reference Bureau works with both public sector and private sector partners to provide information on U.S. and international populations trends and their implications. Its vast database is searchable by topic and region. The user-friendly website also has an extensive list of publications which can be ordered and the downloadable quarterly PRB Reports on America.

UNICEF Monitoring and Statistics [http://www.unicef.org/statistics/index.html]
UNICEF database searchable by region, country, or indicator (adult literacy, mortality rates, etc.).

United Nations Evaluation Forum [http://www.uneval.org]
This is a first stop for evaluation resources related to United Nations programming.

United Nations Millennium Development Goals [http://www.un.org/millenniumgoals/]
This page lists all the millennium development goals and provides a link to the Millennium Indicators Database (click on Statistics on the Achievement of Millennium Goals).

United Way's Outcome Measurement Resource Network [http://www.unitedway.org/outcomes]
United Way's Outcome Measurement Resource Network, US-based but useful guidance and tools including an online resource library.

W.K. Kellogg Foundation [http://www.wkkf.org/default.aspx?tabid=75&CID=281&NID=61&LanguageID=0]
W.K. Kellogg Foundation's very well written handbook on evaluation for non-profits.

World Bank's Independent Evaluations Department [http://www.worldbank.org/oed/]
World Bank's Independent Evaluation Department site includes publications about doing evaluation as well as reports of past evaluations and work on building evaluation capacity in developing countries.

World Food Program (WFP) [http://www.wfp.org/aboutwfp/facts/]
Links to all annual WFP in Statistics documents since 1995, Global Food Aid Statistics published by the International Food Aid Organization System (INTERFAIS), and statistics on WFP procurement activities.

top