 |
 |
 |  |  |
 |
 |
|
|
|
| |
|  | | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
You are here: FANTA-2's Support to Title II Awardees > FAQs
Submit your question to FANTA-2 FAQs for Title II Awardees
MONITORING AND EVALUATION
INDICATORS
1. How many indicators do I need?
2. Where do I find the standard indicator wording and questionnaire items for a given indicator?
4. When is it OK to make changes to indicators and targets?
5. What is the appropriate use and interpretation of the Household Dietary Diversity Score (HDDS)?
GENERAL M&E
6. What is a results framework? Why is it important to M&E?
7. What are my M&E requirements for reporting to Food for Peace (FFP)?
8. Do I need control groups for my baseline study/final evaluation?
10. Should we use Personal Digital Assistants (PDAs) to do our survey?
11. Who should be on my final evaluation team?
SAMPLING
13. Do the same communities need to be sampled at the final evaluation as at the baseline?
15. How do I figure out the sample size?
16. Are we supposed to include Layers in our MYAP proposal?
MCHN
17. How do you deal with severe malnutrition in a PM2A intervention?
18. What about kids over two in a PM2A intervention that are suffering from SAM or MAM?
Didn't see what you were looking for?
MONITORING AND EVALUATION
INDICATORS
1. How many indicators do I need?
There is no set number. Your program needs as many indicators as are necessary and sufficient for cost-effective management and reporting purposes. Collect only the indicators that clearly help meet USAID reporting requirements, manage and improve the program, and tell your program’s story.
2. Where do I find the standard indicator wording and questionnaire items for a given indicator?
Some indicators have standard wording and data collection methodologies. The source for this information depends on the type of indicator.
For Food for Peace (FFP)-required indicators that are fixed as opposed to flexible, the standard wording and methodology can be found in these FANTA-2 publications, available on the FANTA-2 website:
For indicators that come from USAID Missions—the “F” Standard indicators—the definitions (methodologies are not given) can be found at https://communities.usaidallnet.gov/fa/node/176.
Most maternal and child health and nutrition (MCHN) outcome or impact indicators have internationally used standard formulations and methodologies, such as the Demographic and Health Surveys (DHS), which can be found at http://www.measuredhs.com/aboutsurveys/methodology/instruments.cfm.
Other sources for health indicators include:
The World Health Organization’s Indicators for Assessing Infant and Young Child Feeding Practices - Part I: Definitions
USAID’s Knowledge, Practices, and Coverage (KPC) Modules for 2009 Grantees
A helpful source of ideas for poverty indicators is USAID’s Poverty Assessment Tools website.
3. Should an Indicator Performance Tracking Table (IPTT) include all of a program’s indicators or just the more substantive ones?
The IPTT is a tool that Awardees use to manage their set of indicators. The IPTT must at least have all of the USAID-required indicators, including those from the Missions and FFP. But to be useful as a program management tool, it should include all of the indicators that a program measures.
4. When is it OK to make changes to indicators and targets?
Official Food for Peace (FFP) guidance is silent on this question. However, based on common practice and FANTA-2’s recommendation, indicators should be refined at program start-up, before baseline data collection. Targets and baseline values should be revised after baseline data become available. After that, programs should assess their M&E system’s performance once a year. If an indicator is ill-conceived or targets are widely off the mark, the program should ask FFP for permission to make a change in their Pipeline Resource Estimate Proposal (PREP) for the coming year. Awardees must show how the change makes their M&E system more effective.
5. What is the appropriate use and interpretation of the Household Dietary Diversity Score (HDDS)?
The HDDS is an indicator required by Food for Peace that measures household access to food, one of the components of food security. While the indicator sounds as if it measures the nutritional quality of a household’s diet, it is not designed to be used that way. It is a proxy for a household’s socioeconomic status. Households that are better off eat a more diverse diet. In other words, a higher HDDS corresponds to a higher socioeconomic level and greater food security.
GENERAL M&E
6. What is a results framework? Why is it important to M&E?
A results framework (RF) is an explicit theory or model of how a program’s activities will cause the intended or observed outcomes. It’s a roadmap of what the program will accomplish and how. Using an RF to plan a project helps to ensure a clear and unique objective, keep the focus on the objective rather than the activities, and ensure that a solid, evidence-based, theory of change underlies the choice of activities. Typically, the framework is done as a graphic supplemented by narrative.
7. What are my M&E requirements for reporting to Food for Peace (FFP)?
For reporting requirements, see FFP Information Bulletin 09-07: Title II Awardee Reporting Requirements.
8. Do I need control groups for my baseline study/final evaluation?
Food for Peace does not require control groups for baseline studies and final evaluations. Control or comparison groups are discouraged because of their added cost and complexity.
9. What is the difference between the people measured at baseline survey and final evaluation, and the people measured annually in the Multi-Year Assistance Programs (MYAPs)? What should be done for the Mid-Term Evaluation (MTE)?
The baseline survey and final evaluation are done at the population level; annual monitoring is done at the beneficiary level. This means that at baseline and final, the sampling universe is the entire target population living in the areas covered by the MYAP. For example, if your program targets vulnerable households with children under 2, you would sample from among all vulnerable households with children under 2, whether or not they participated in your program. For annual monitoring, the program measures only the program’s direct beneficiaries. The MTE does not need to be population-based or quantitative in nature. A participatory qualitative assessment is encouraged.
10. Should we use Personal Digital Assistants (PDAs) to do our survey?
Some programs use handheld computers or PDAs to collect survey data. You have to weigh the pros and the cons. PDAs simplify some tasks but complicate others. On one hand, using PDAs reduces errors in data collection and doesn’t require a separate data entry step. On the other hand, enumerators and supervisors must be trained to use PDAs and laptops (for the supervisors), and one or two people are needed to program the questionnaire into the PDA software.
11. Who should be on my final evaluation team?
The evaluation team should include:
A clear team leader with expertise in all aspects of conducting quantitative surveys and program evaluations. The team leader might also have expertise in one of the technical areas relevant to the Multi-Year Assistance Program (MYAP).
Sectoral leaders, based on the project’s technical components (e.g., maternal and child health and nutrition (MCHN), agriculture, small enterprise, water).
At least one person with expertise in qualitative methods
Supervisors and enumerators in the field
The final evaluation must be external, so the team leader and the sectoral experts must not be employed by or affiliated with the Awardee. MYAP staff can participate in designing the evaluation, providing information to the evaluators, collecting data, and reviewing and commenting on the final report. But the evaluators are responsible for analyzing the data and writing the final report.
12. What's the difference between the Standardized Annual Performance Questionnaire and the Indicator Performance Tracking Table (IPTT)?
The SAPQ is a form that programs use to report annually on FFP-required indicators. It is used to meet FFP’s reporting needs, not manage individual MYAPs. FFP’s Program and Technical Division compiles the data from the SAPQs to use in its reporting to Congress. The SAPQ is not reviewed by the Agreement Officer’s Technical Representative (AOTR). The IPTT has all the program indicators, not just FFP-required ones, and is used for program management. The IPTT is reviewed by the MYAP’s AOTR.
SAMPLING
13. Do the same communities need to be sampled at the final evaluation as at the baseline?
For the baseline and final evaluation to be comparable, they must be drawn from the same sampling universe. That means both surveys use the same “list” or “sampling frame” (updated with new population numbers if possible). The sample that is actually drawn will probably be different between baseline and final. That is up to chance.
14. If I used one sampling approach at baseline and a different one at final evaluation, are the two studies still comparable?
If both sampling approaches are statistically valid and drawn from the same sampling frame, they should be comparable. Both the baseline and final evaluation sampling methods aim to represent the same target population.
15. How do I figure out the sample size?
The FANTA-2 Sampling Guide provides a step-by-step explanation and sample size formulas.
16. Are we supposed to include Layers in our MYAP proposal?
No. Layers is a tool for USAID Missions to use to monitor Title II programs. Awardees will participate when Layers takes place in their country, but they do not implement it.
MCHN
17. How do you deal with severe malnutrition in a PM2A intervention?
In areas where SAM is a problem, PM2A programs should be closely linked with SAM treatment services and refer children with SAM to those services. If there are no SAM treatment services, a MYAP may want to consider establishing a CMAM program, this is not part of PM2A, but part of the MYAP’s overall MCHN component.
18. What about kids over two in a PM2A intervention that are suffering from SAM or MAM?
Children over two will not be enrolled in a PM2A intervention. However, in an area with high SAM prevalence, the MYAP may choose to implement screening, referral and/or treatment services for children over two who have SAM. Please note: PM2A is part of a MYAPs overall MCHN component, which can comprise several other interventions.
Didn't see what you were looking for?
