Immpact Toolkit: Module 4
Version 2
Evaluation tools
INTRODUCTION
This module contains the evaluation tools developed by Immpact that have been introduced in the previous three modules. The different tools can be used within a composite evaluation of a safe motherhood programme, or can be used independently to measure specific elements of a programme.
The nine different tools are summarized in table 4.1. As the table shows, the tools evaluate various different components of a safe motherhood programme, including the measurement of maternal mortality, the measurement and description of the quality of health services, evaluation of health systems factors, and evaluation of economic outcomes.
Table 4.1: Tools contained in Module 4
| Tool name and number | Tool type | Description |
|---|---|---|
| 1. Sampling at Service Sites (SSS) | Measurement of maternal mortality | An approach to capturing data on maternal deaths where researchers visit locations where women gather in large numbers, such as health facilities and markets. |
| 2. Rapid Ascertainment Process for Institutional Death (RAPID) | Measurement of maternal mortality | A method to improve monitoring of pregnancy-related deaths in hospitals by retrospectively identifying deaths that have been missed during routine reporting. |
| 3. Tracing adverse and favourable events in pregnancy care (TRACE) | Measurement and description of maternal mortality, morbidity and the quality of maternal health care services | An enquiry to trace adverse and favourable events in pregnancy care. Gathers qualitative information on care for cases of maternal death, cases of severe, short-term illness or disability (near misses), and if required, normal cases and other complications. |
| 4. Perceptions of Quality of Care (PQOC) | Measurement and description of the quality of maternal health care services | Qualitative methods employed to study perceptions of community members and providers regarding barriers and facilitators to good quality of care. Provides contextual information about factors that may affect skilled care at delivery. |
| 5. Health Worker Incentives Survey (HWIS) | Evaluation of health systems factors | Questions to survey motivational factors in health workers and other aspects of human resource contexts in order to feed into the functionality measurement of facilities (or community midwifery services in Indonesia). |
| 6. Productivity Costs Survey | Evaluation of economic outcomes | Household interviews that focus on the impact of maternal ill health on productivity. Provides better evidence of the economic impact of different health care interventions.|
| 7. Household Costs Survey | Evaluation of economic outcomes | A household questionnaire that assesses the costs of maternal health paid by households. |
| 8. Outcomes after Pregnancy (OAP) | Measurement of maternal morbidity | An interdisciplinary approach to explore the interrelated social, psychological, physical and economic consequences of complications during pregnancy and childbirth. |
| 9. Maternal Death from Informants (MADE-IN) and Maternal Death Follow-on Review (MADE-FOR) | Measurement of maternal mortality | Research tool that uses informants to identify pregnancy-related deaths, in order to estimate the maternal mortality ratio and rate and possibly provide information on the causes of mortality (MADE-IN). Follow-up interviews with family members of the dead women (MADE-FOR) confirm if deaths are maternal or non-maternal and explore the cause of and circumstances surrounding the death. |
As discussed in Module 1, these nine tools are the first in a series of Immpact tools to be made available for wider use. Other Immpact tools under development will be added in future editions of this toolkit. These other tools are described in table 1.2 of Module 1. In addition, tools used by Immpact that are available elsewhere are described in table 1.3 of Module 1.
Each tool in this module contains introductory material on the study, outlines what the tool can be used to evaluate, discusses the limitations of using the tool, gives a step-by-step guide of how the tool can be used, and presents the resource implications of using it. The specific data collection instruments used for the study are also contained in the tools.
The tools presented are examples used from Immpact evaluations in Burkina Faso, Ghana, and Indonesia. Some of the research instruments contained in the tools have been adapted to a more 'generic' format, while others remain specific to the country where they were used. Researchers using these tools will need to adapt the research instruments for use within their own evaluation or sub study. More information on adaptation is contained in Module 5, Technical Annex C.
The tools presented in this module are a selection of maternal health evaluation methodologies developed by Immpact. Technical Annex F in Module 5 also lists additional tools developed by other research institutions that can also be of great value in safe motherhood evaluations.
Finally, the last section of this module contains a technical paper on the combined use of three methodologies within an Immpact evaluation in Ghana. The paper, Evaluating a health financing policy change: framework and suggested approaches presents an approach to evaluating the health systems component of the Ghana evaluation question, using the Health Worker Incentives Survey (Tool 5) and two other methodologies which are not stand-alone tools; the key informant interviews and financial flows tracking.