OJEpi  Vol.9 No.3 , August 2019
How to Develop a Realist Evaluation to Evaluate the Effectiveness of Integrated Management of Childhood Illness Strategy in Community Care Sites in the Democratic Republic of Congo
ABSTRACT
Background: In sub-Saharan Africa, the DRC ranks among countries with high infant mortality. To solve this problem, the Health Ministry has set up a programme: the use of integrated management childhood illness (IMCI) in community care sites (CCSs) managed by non-health professional persons known as community health workers (CHW). This approach has not yet been integrated into the health pyramid. This is why we chose the Realist Evaluation Approach to evaluate the effectiveness of this programme on Universal Access to care for children under 5 years of age in the DRC. Methods/design: To conduct this exploratory study which is essentially a descriptive case study, we applied the principles of a realist evaluation by context—mechanisms—effects explained in the conceptual framework. The preliminary theory of the programme was elaborated by data collection through a literature review coupled with semi-structured interviews with key informants who were managers of the government programme and technical and financial partners of the programme. Results: IMCI has been evaluated and shown its positive contribution in the reduction of global children mortality. In the DRC, summaries of data collection through a literature review coupled with semi-structured interviews showed the same result. However, under criticism, based on the data collected during this exploratory study, it is promisingly apparent that this programme will achieve the objectives assigned to it through the preliminary theory of the programme. Conclusion: The literature review corroborated the effectiveness of the IMCI programme. The application of this programme in CCS with non-health professionals who are Community Health Workers has been shown to be effective in the DRC. Based on this reality, it was possible to define the basis for a realist evaluation of the effectiveness of the programme in CCS.

1. Background

The Democratic Republic of the Congo (DRC), like other developing countries in Sub-Saharan Africa (SSA) has a limited health situation for the general population and for children under 5 years of age. According to recent health statistics, the DRC ranks among the six countries in the world that contribute to 50% of the weight of global infant mortality. It ranks successively after India, Nigeria, China, Pakistan and Ethiopia [1] .

The last demographic and health survey carried out in the DRC in 2014 showed that: neonatal mortality was 42‰; infant mortality 58‰; and infant and children mortality 104‰. A percentage of between 60% and 80% of these children died in the community from preventable causes, often as a result of diseases preventable by immunization, such as malaria, diarrhea and pneumonia associated with malnutrition [2] .

However, there are some effective strategies to prevent and treat sick children. Nevertheless, access to these interventions is extremely limited when available as children under the age of five often do not reach them. One of the reasons is that health services are often too remote or too costly.

One of the strategies implemented to improve this situation is the integrated management of childhood illnesses (IMCI) that contributes to universal access to care [3] [4] .

The universal access to care strategy advocates that for wider coverage, the provision of quality care can be based on three pillars: 1) The first pillar is supply, the State must provide the infrastructure, equipment (medical and pharmaceutical) and adequate quality and quantity of staff to provide care to the concerned population; 2) The second pillar is the removal of obstacles namely financial, cultural, and geographical to facilitate access to health care; and 3) The third pillar is equity and financial support for vulnerable populations [5] [6] .

Since 2005, this strategy has been progressively introduced in community care sites (CCS) in the DRC. The community care site is a space that covers a locality, one or more villages where households are located further than a 5 km radius from a health centre (HC) or separated by a natural barrier; provides access to first aid before orientation to a health centre or a home care monitoring of children under 5 years of age by a community health worker (CHW).

The community health worker is a non-professional health member of the village who is given training on the management of the symptoms of the following major diseases: malaria, diarrhea and pneumonia [7] .

In 2017, the number of community care sites was recorded as 6789 and were operational in 402 of the 516 DRC’s health areas.

However, this approach always has been subject to several questions. To this end, two trends have emerged since the implementation of this programme: 1) the approach in favor of this. In 2001, it was noted that the IMCI coordination meeting held in Santo Domingo that an improvement in the quality of services in the health area training was indispensable, it did not contribute significantly to the reduction in child mortality. An IMCI community approach was recommended to address this situation; and 2) the approach against this. In 2013, a study conducted in the DRC on the establishment of the health care community to reduce geographical inaccessibility indicated that there were enormous risks because the number of qualified and motivated staff members for regular monitoring and supervision of non-health care professionals committed to the provision of healthcare was limited [8] .

To take a position in relation to these two trends, we propose to evaluate the effectiveness of the use of IMCI in CCS on universal access to care (UAC) from children under 5 years of age in the DRC by using the realist evaluation approach. Moreover, this study proposes a conceptual framework model for an evaluation of this complex programme using the realist evaluation approach in a context like that of the DRC. The realist evaluation approach is a method proposed by Pawson and Tilley in 2005 that starts from the context—mechanisms-effects to evaluate social programmes in a complex system [9] . The aforementioned authors argued that the effectiveness of an intervention relied on the underlying mechanisms implemented in a given context. Furthermore, two key concepts that distinguish approaches based on the theory driven evaluation with traditional evaluation approaches: 1) the influence of context on programme outcomes; and 2) the use of a mechanistic rather than a counterfactual approach to determine causality [10] [11] .

This study’s general objective is to describe the fundamental principles for laying the groundwork for a realist evaluation in order to evaluate the effectiveness of the use of IMCI in community care sites in the DRC.

On the basis of the main constituents of the context—mechanisms—effects approach (CME), three specific objectives have been defined:

1) to describe the context in which this programme (IMCI) was introduced to the health system in the DRC;

2) to identify all mechanisms (input, process) set up for the implementation of this programme; and

3) to compare the results (output, effect) expected after implementation in relation to the initial objectives of the programme.

Starting from these specific objectives, a conceptual framework model is proposed as shown in Figure 1.

Figure 1. Conceptual framework model to evaluate IMCI in Community Care Sites in the Democratic Republic of Congo. Source: Authors, adapted from Pawson and Tilley model.

The conceptual framework model which we developed explains the logic of a realist evaluation that is summed up in three fundamental concepts namely context, mechanisms and effects.

In the DRC, the health context is summed up by the health situation which is marked by a high infant mortality. This situation led the health to integrate in the country’s health policy IMCI as a strategy to reduce infant mortality. The integration of IMCI in the minimum activity package of the health area (HA), the organization and implementation of community care sites in difficult to access health areas had an influence on the elements of the healthcare system through: 1) Building professional capacity health and non-health professional such as community health workers; 2) The organization, management and input endowments of community care sites for case management and 3) Community Health Workers family and community visits to promote key family best practices for the well-being of children.

Taking into account individuals and households characteristics, the health system influence enriched community involvement with community support for community care sites activities that improve behavioral change through communication for social and behavioral change that should improve offer and demand of IMCI services.

This shown that logically we should have an improved use of IMCI services by children under 5 years of age.

Then, if improving the quality of care for children under 5 years of age by offered quality services at community care sites, the use of IMCI in community care sites will be supported in the health system.

Thus, with the application of this model we can predict that this approach can impact the DRC’s health system by proposing its institutionalization in the health pyramid.

The interactions between the context and all mechanisms used to achieve the expected effects of this programme are explained in this conceptual framework model and summarize the medium-range theory developed by Valéry Ridde, Bruno Marchal and al in 2012 [12] [13] .

2. Methods/Design

This exploratory study is essentially a descriptive case study. We applied the principles of a realist evaluation. The data were collected through a literature review coupled with semi-structured interviews with key informants namely managers of the government programme and technical and financial partnership of the programme.

The participants in this study were selected on the basis of a reasoned choice. A total of 10 program managers at the Health Ministry; and 10 from technical and financial partnership were interviewed.

The University of Kinshasa Ethic Comitee approved all protocols of our research on April 2018.

Then a written informed consent was obtained from all study participants before semi-structured interviews.

Data was collected on May 2018 to October 2018.

To summarize our literature review, a critical analysis grid was proposed. See Figure 2.

This critical analysis grid of IMCI articles was made to identify the positive effects of IMCI integration in developing countries around the world on reducing the morbidity and mortality of children under five years of age. It contains 8 analysis criteria applicable to all selected articles related to the evaluation of IMCI namely: The study objective; the study design; the factors studied, the judgment criteria; the sampling and study population; most confounding factors and biases; the study result and conclusion.

For this purpose, it was necessary to answer three essential questions: Is it possible to find in the article the information on the evaluation criteria?, the way in which the evaluation criteria were addressed? and what is the study validity?

From 35 articles analysed, 25 were selected to form part of the summary of our literature review for the integration of IMCI in the DRC’s health system. As for the semi-structured interview with program managers at the Health Ministry and technical and financial partnership, a questionnaire was elaborated that study 3 variables: 1) the context in which IMCI strategy was introduced in the DRC’s health system; 2) the process of implementation and 3) expected results.

This semi-structured interview with the above-mentioned stakeholders showed globally that it is in a context of high infant and child mortality, into account taking the country’s poor health coverage in terms of quality infrastructure and the lack of access to quality care; that the Health Ministry has resolved the integration of IMCI in CCS.

All data for this step are summarized in the following paragraph as results.

Figure 2. Critical analysis grid of IMCI evaluation articles to identify positive effects of IMCI to include in our literature review. Source: Authors.

3. Results

The results of our study are itemised, as indicated by methodology in two points: 1) the summary of the literature review; and 2) that of the key informants’ interview that will end with the formulation of the preliminary theory of the programme. We describe the results in the following paragraphs:

3.1. Summary of Information Collected from Literature Review

During 2000, a Tanzanian quasi-experimental study of IMCI implementation showed that IMCI-trained health workers in the intervention districts correctly treated 65% of children aged under five years while this rate was 16% for non-IMCI-trained health workers in comparison districts. This evaluation showed that IMCI had contributed to reducing under-five mortality by 13% in the districts where the initiative was implemented [14] [15] .

In April 2000 in Morocco, an evaluation of the pilot phase of a quasi-experimental study of IMCI showed that the children living in the districts (Agadir and Meknes) where the IMCI was implemented, received better care than those in comparison districts (Tetouan and Larache). The authors of this report recommended that the Moroccan Ministry of Health take into account the context of implementation (availability of human and material resources and improvement of interventions to support trained agents) during the extension of the IMCI to other provinces of Morocco [16] [17] .

In 2001 in Peru, the evaluation of the pilot phase with a quasi-experimental study also showed that children are better cared for in IMCI regions compared to non-IMCI regions. However, it has been shown that the evaluation of general signs of danger, the scourge of malnutrition, the correct prescription of antibiotics and malaria drugs still require considerable improvement [18] .

In 2003, the same observation was made in Zambia by an analytical review of the IMCI, which concerned several countries implementing the IMCI. This was a qualitative and single-measure estimate study jointly conducted by the Department for International Development (DFID), the United Nations International Children’s, Emergency Fund (UNICEF), the United States Agency for International Development (USAID) and the World Health Organization (WHO). According to the authors of aforementioned analytical review, a weak impact of IMCI results from several factors that include: insufficient funding in response to increased disease costs; limited human resources; gap policies; lack of protection for the most vulnerable; and the lack of transport infrastructure in a dispersed territory.

Surveys in Uganda, Republic of South Africa, Malawi, Tanzania and Bangladesh have also shown that a good implementation of the third component of IMCI (improvement of key family and community practices in child health) allows communities to adopt key practices that improve nutrition, disease prevention, and home care [19] .

In 2012, WHO and UNICEF made a joint statement based on evidence that considers the use of IMCI-C by community health workers as an equity strategy to improve access to essential care for children aged under 5 years. Community health workers represent an effective investment solution to strengthen primary health care. However, community management of cases should not be considered as a low-cost measure. Effective implementation requires political support, skill-building, on-going support through formative supervision, performance maintenance and regular supply of essential supplies and medicines for proper management of malaria, diarrhoea and pneumonia in the community [20] .

The aforesaid statement was based on some experiences in the following countries: 1) In Ghana, 92% of parents of sick children sought care from community agents trained in the management of pneumonia and malaria. Most of these children were cared for within 24 hours after the onset of fever, the first 24 hours crucial for the vital prognosis of children if nothing is done; 2) In Zambia, a study on integrated community management of pneumonia and malaria revealed that 68% of children with pneumonia and malaria received early and appropriate treatment from community health workers; and 3) In Malawi, 68% of the classifications of common diseases (malaria, pneumonia, malnutrition) by community health workers were consistent with assessments made by physicians and 63% of children benefited from appropriate drugs [21] .

In 2017, a study conduct in Mozambique also showed that community health workers trained and supervised regularly; regardless of their instruction level; are able to provide adequate first aid in their communities and provide the services needed by hard-to-reach populations.

Because they have confidence in their abilities and are highly valued by the communities they deserve and most of them are motivated to continue as volunteers without regular compensation. [22] .

In 2010, the principles of theory-based evaluation were applied as part of an ex-post evaluation of one of the strategies of the PASSAGE programme, a project of solidarity approach to reproductive health. PASSAGE is a three-year EU-funded intervention to improve the continuity of urban reproductive health care in Mopti (Mali), Maroua (Cameroon) and two districts of Ouagadougou (Burkina Faso). This evaluation allowed the creation of networks between public and private health and social service providers in adolescent’s sexual and reproductive health. These networks have helped to improve the integration of services and continuity of care for adolescents [23] .

In 2018, a theory of change approach was conducted in the DRC to evaluate the change observed after the implementation of the PHAC project (Access to Primary Health Care) with the support of DFID and showed that if the programme was conceived as context, mechanisms and effects including the result chain (input, process, output, outcome, effect and impact) are well defined and the necessary means for implementation are available with the involvement of all stakeholders; then the change expected can happen after the implementation of the programme [24] .

3.2. Summary of Information Collected from Key Informants

The information gathered from key informants through a semi-structured interview and the review of the database of different programmes involved in the IMCI in the DRC, namely the National Malaria Programme, the National Diarrhoea Programme and the National Acute Respiratory Infections Programme are detailed in the following paragraphs in the form of context—mechanisms—effects. They are summarized by the context in which the IMCI in community care sites was introduced; the implementation process which takes into account the minimum activity package of a health area, the community involvement component, the request and offer of care; the expected effects after the implementation of the programme by considering the use of services and the quality of management.

3.2.1. Context

In DRC, the health situation of the general population and that of children is marked by high child mortality (DHS 2014: Demographic and Health Survey), in particular: neonatal: 42‰ NV, infantile: 58‰ NV and infant and juvenile: 104‰ NV.

It should be noted that the mortality rate is 80% for children under the age of five in the community. In addition to these red indicators, there is a low health cover (30%), and a low use of services (35%). Only 35% of the population are within a 5 km radius of a health service; and a mere 15% of the population are able to access quality health care financially.

In order to resolve this situation, some reforms have been put in place in the DRC: 1) in macroeconomic terms the achievement of the Heavily Indebted Poor Countries (HIPC) initiative has allowed debt cancellation [25] . The costs of repaying the external debt were directed towards the basic social sectors; 2) in the context of sectorial reform, the Growth Strategy and Poverty Reduction Document (GSPRD) was developed. The contribution of the health sector to the implementation of the (GSPRD) was carried out through the Health System Strengthening Strategy (HSSS), from which the various National Health Development Plans (NHDPs) were derived; 3) Since 2006, the government of the DRC has adopted a Government’s Priority Action Programme (GPAP) in which the strategic orientations for each sector of public life are found and the resources allocated to it to reach the Millennium Development Goals (MDGs); and 4) In 2013, an assessment of the achievement of the MDGs was made, which enabled the implementation of Framework for Accelerating the Achievement of MDGs 4 and 5 (FAM 4, 5) with the following interventions [26] :

1) Prevention and promotion of essential family practices;

2) The integration of IMCI in community care sites to support: malaria, diarrhoea and pneumonia;

3) Immunization;

4) Assisted births and new-borns care; and

5) Nutrition, family planning, HIV/AIDS and TB control [27] .

3.2.2. Mechanisms

In order to achieve the second component that relates to the integration of IMCI-C in community care sites, the following actions have been carried out in the form of the programme implementation process:

On the strategic plan, the Health Ministry, after expressing needs and planning with the community, adopted the IMCI strategy in the DRC’s health policy. For example, the Ministry’s experts in the field of IMCI have developed a roadmap for the implementation of this programme. Moreover, some tools that have been developed include community care sites implementation guides, and IMCI modules. A mobilization of resources was made with the support of the technical and financial partnership for the implementation of the programme [28] .

At the intermediate level, members of the Provincial Framework Teams (PFT) were mobilized for the validation and consolidation of the various tools and indicators at the Provincial Steering Committee Health Sector (PSC-HS) level. They attended health areas during the implementation of the programme to ensure the monitoring of data collected at the community care sites, the management of the inputs, the payment of the performance premiums to the health areas and community care sites with technical and financial partnership [29] .

At the operational level, members of the Health Areas Framework Team (HAFT) were used to recruit community health workers with support from the community. The health area also identified community care sites, the monitoring of capacity building of all providers involved in the programme, and the formal installation of community health workers and the staffing of CCS in inputs to carry out the support effectively and gratuitous cases of simple malaria, diarrhoea and pneumonia according to the national protocol. This support ends with the orientation of complicated cases to the health centre or the return home followed by a regular visit of the case by a community site relay. Community health workers have also the role of recovery of lost children for immunization and distribution of long lasting insecticidal net (Permanet) in the community [30] .

The monitoring of children’s quality care in the CCS is done through the visits carried out by a senior nurse (sometimes accompanied by members of the health area framework team and technical and financial partnership) on the community health worker.

Data generated by CCS are reported in a register and then encoded in the National Information System through the DHIS2 software.

3.2.3. Effects

The effects of the implementation of the programme are expressed in the form of expected results: 1) correct management of single malaria cases, diarrhoea and pneumonia for children under 5 years of age at the CCS, excluding children under 2 months of age and pregnant women; and 2) an improvement in key family practices in the community with a view to contributing to the reduction of mortality in the DRC [31] [32] .

Reference data that we will use as baseline to evaluate the effectiveness of IMCI at the community care sites were derived from the database of three programmes involved in IMCI. They indicate that from 2015 to 2016 morbidity in these three diseases has seen a significant improvement in the detection, diagnosis and case management at the CCS. These data are shown in the graphs, namely Figures 3-5, as follows:

The analysis of these three graphs shows us that from 2015 to 2016, efforts were made to improve the management of the three IMCI diseases at the community care sites through case diagnosis and treatment. During 2015, the percentage for cases supported versus those in 2016 were: malaria (94% vs. 95%); diarrhoea (93% vs. 95%); and pneumonia (94% vs. 95%).

The analysis of the results we obtained after our literature review and the key informants’ interview ended with the formulation of the preliminary theory of the programme. Indeed, in 2012 at the end of a state of health, in order to achieve the Millennium Development Goals by 2015, the DRC’s Health Ministry set up some strategic axes, including the framework for accelerating the achievement of Millennium Development Goals 4 and 5.

Figure 3. Malaria treated cases at Community Care Sites use as baseline to evaluate the effectiveness of IMCI in the Democratic Republic of Congo. Source: Malaria Programme.

Figure 4. Diarrhoea treated cases at Community Care Sites use as baseline to evaluate the effectiveness of IMCI in the Democratic Republic of Congo. Source: Diarrhoea Programme.

Figure 5. Pneumonia treated cases at Community Care Sites use as baseline to evaluate the effectiveness of IMCI in the Democratic Republic of Congo. Source: National acute respiratory infections Programme.

The preliminary theory of programme formulated reads as follows:

“The integration of IMCI in CCS managed by Community Health Workers is an effective intervention to contribute to the reduction of juvenile and infant mortality due to malaria, diarrhoea and pneumonia in DRC.”

4. Discussion

Since the beginning of its implementation around the world in 1990, IMCI has been the subject of evaluations by several individuals and bodies, including the Department for International Development (DFID), the World Health Organization (WHO), the United Nations International Children’s, Emergency Fund (UNICEF), the United States Agency for International Development (USAID). These evaluations have been carried out nationally or multi-nationally and have shown the positive contribution of this approach to the reduction in infant and child mortality. This is the case for Bangladesh, Brazil, Peru, Tanzania and Uganda [33] [34] .

In DRC, all information we collected during interviews and consulting DHIS2 software of Health Ministry after implementation of this programme shown the same positive contribution of this approach.

Based on the data collected during this exploratory study, it is apparent that the integration of IMCI in CCS in the DRC is promising to achieve the objectives assigned to the programme.

However, a study conducted in 2013 in the DRC found that the establishment of community care sites to reduce geographical inaccessibility posed enormous risks. The number of agents qualified and motivated to ensure regular monitoring and supervision of non-health professionals committed to the supply of care is limited. It would be a good idea to stop setting-up community care sites, pending a rigorous assessment of their impact. Only those community care sites that already have rigorous supervision of a health area framework team should continue to operate [35] [36] .

Thus, when one takes into account the elements of the literature review and on the basis of the projection made on this approach through the reading of our conceptual framework model as we have explained in the methodology, we propose that the model set the groundwork for a realist evaluation of this programme.

5. Conclusion

The literature review corroborated the effectiveness of the IMCI programme. The application of this programme at CCSs with non-health professionals who are community health workers has been shown to be effective in the DRC. Based on this reality, it was possible to define the basis for a realist evaluation of the effectiveness of the programme at CCSs. The other phases of this research will lead to the formulation of the final theory of the programme, after the verification of the preliminary theory. This approach will be carried out with the collection and analyses of data from field implementer actors, beneficiaries and programme technical and financial partnership to measure the effectiveness of this programme as a contribution to universal access to care for children under the age of five years in the DRC.

Ethics Approval and Consent to Participate

Written informed consent was obtained from all study participants. The University of Kinshasa Ethic Comittee approved all protocols (Reference number: ESP/VD/DIR/CE/223/KL/2018).

Availability of Data and Material

All data generated or analysed during this study are included in this published article [and its supplementary information files].

Authors’ Contributions

JME contributed to the design of the evaluation protocol, the actual evaluation, the analysis of findings and the writing of the manuscript,

DNB contributed to the protocol development and the writing of the manuscript,

PKK contributed to the design of the protocol evaluation,

FKN contributed to the protocol development.

Acknowledgements

The authors thank Bart Criel for guiding the corresponding author in the choice of the realist evaluation approach to this study and F. E. Meyer for English language editing of the manuscript.

List of Abbreviations

CCS: Community Care Site

CME: Context-Mechanisms-Effects

CHW: Community Health Worker

DFID: Department for International Development

DHS: Demographic and Health Survey

DHIS 2: Data Health Information System 2 software

DRC: Democratic Republic of the Congo

FAM 4, 5: Framework for Accelerating the Achievement of MDGs 4 and 5

GSPRD: Growth Strategy and Poverty Reduction Document

HAFT: Health Area Framework Team

HIPC: Heavily Indebted Poor Countries

IMCI: Integrated Management Childhood Illness

IMT: Institute of Tropical Medicine Antwerp

MDGs 4, 5: Millennium Development Goals 4 and 5

NHDP: National Health Development Plan

PFT: Provincial Framework Teams

PSC-HS: Provincial Steering Committee Health Sector

HSSS: Health System Strengthening Strategy

UNICEF: United Nations International Children’s, Emergency Fund

USAID: United States Agency for International Development

WHO: World Health Organization

Cite this paper
Etshumba, J. , Bebe, D. , Kalambay, P. and Nappa, F. (2019) How to Develop a Realist Evaluation to Evaluate the Effectiveness of Integrated Management of Childhood Illness Strategy in Community Care Sites in the Democratic Republic of Congo. Open Journal of Epidemiology, 9, 243-258. doi: 10.4236/ojepi.2019.93019.
References
[1]   Black, R.E., Cousens, S., Johnson, H.L., et al. (2010) Global, Regional, and National Causes of Child Mortality in 2008: A Systematic Analysis. The Lancet, 375, 1969-1987.
https://doi.org/10.1016/S0140-6736(10)60549-1

[2]   Plan Ministry, Public Health Ministry, ICF International (2013) Demographic and Public Health Survey (DHS) in the DRC.

[3]   World Health Organization (2009) World Health Report: Primary Health Care “Now More than Ever”. WHO, Geneva.

[4]   Lambrechts, T., Bryce, J. and Orinda, V. (2000) Integrated Management of Childhood Illnesses: Synthesis of Early Experiences. Bulletin of the World Health Organization, Collection of Articles, Geneva.

[5]   WHO (2014) Strategy for Universal Access to Health and Universal Health Coverage. 53rd Governing Council 66th Session of WHO Regional Committee for the Americas, Washington DC, 87-128.

[6]   Public Health Ministry (2016) Procedures Manual of Community Structures and Approaches, DRC.

[7]   Dawson, P., et al. (2008) From Research to National Expansion: 20 Years of Community-Based Management of Childhood Pneumonia in Nepal. Review of the World Health Organization, 86, 339-343.
https://doi.org/10.2471/BLT.07.047688

[8]   Mawazo, G. (2014) Implementation of Community Care Sites in the Democratic Republic of Congo: Consecration of a Double Standard in Access to Care. Pan African Medical Journal, 14, 158.

[9]   Pawson, R. and Tilley, N. (1997) Realistic Evaluation. Sage Publications Ltd., London.

[10]   Armstrong, S.J.R., Adam, T., Mshinda, H., et al. (2004) Effectiveness and Cost of Facility-Based Integrated Management of Childhood Illness (IMCI) in Tanzania. The Lancet, 364, 1583-1594.
https://doi.org/10.1016/S0140-6736(04)17311-X

[11]   Macklin, R. (2004) Double Standards in Medical Research in Developing Countries. Cambridge University Press, Cambridge.
https://doi.org/10.1017/CBO9780511495359

[12]   Marchal, B., van Belle, S., van Olmen, J., et al. (2012) Is Realist Evaluation Keeping Its Promise? A Review of Published Empirical Studies in the Field of Health Systems Research. Evaluation, 18, 192-212.
https://doi.org/10.1177/1356389012442444

[13]   Valery, R. and Van Olmen, R.E. (2012) The Realist Approach to the Real Test of Program Evaluation. Canadian Journal of Program Evaluation, 18, 91-113.

[14]   World Health Organization (2002) Multi-Country Evaluation of the Effectiveness, Cost and Impact of IMCI (ECM). Progress Report, Department of Health and Child and Adolescent Development, Geneva.

[15]   Nshimyumukiza, L. (2009) Evaluation of the Implementation of the Integrated Management of Childhood Illness (IMCI) in Kirehe District, Rwanda, Quebec.

[16]   Huicho, L., Davila, M., Gonzales, F., et al. (2005) Implementation of the Integrated Management of Childhood Illness Strategy in Peru and Its Association with Health Indicators: An Ecological Analysis. Health Policy and Planning, 20, i32-i41.
https://doi.org/10.1093/heapol/czi052

[17]   Denis, J.L. and Champagne, F. (2000) Implant Analysis: Models and Methods. Canadian Journal of Health Program Evaluation.

[18]   Bryce, J., Victora, C.G., Habicht, J.P. and Vaughan, J.P. (2016) The Multi-Country Evaluation of the Integrated Management of Childhood Illness Strategy: Lessons for the Evaluation of Public Health Interventions. American Journal of Public Health, 94, 406-415.
https://doi.org/10.2105/AJPH.94.3.406

[19]   WHO, UNICEF (2012) Common Statement: Management of Childhood Illness Cases in the Community: An Equity Strategy to Improve Access to Essential Health Care for Children under Five. Geneva.

[20]   World Health Organization (2002) The Multi-Country Evaluation of IMCI Effectiveness, Cost and Impact (MCE): Progress Report, May 2001-April 2002.
http://www.who.int/iris/handle/10665/67381

[21]   Public Health Ministry (2012) Integrated Management of Childhood Illnesses in Community: Best Practices and Bottlenecks in Program Implementation Documentation in the DRC, Kinshasa.

[22]   Guenther, T., Sadruddin, S., Finnegan, K., et al. (2017) Contribution of Community Health Workers to Improving Access to Timely and Appropriate Case Management of Childhood Fever in Mozambique. Journal of Global Health, 7, Article ID: 010402.
https://doi.org/10.7189/jogh.07.010402

[23]   Van Belle, S.B., Marchal, B., Dubourg, D., et al. (2010) How to Develop a Theory Driven Evaluation Design? Lessons Learned from an Adolescent Sexual and Reproductive Health Program in West Africa. BMC Public Health, 10, Article No. 741.
https://doi.org/10.1186/1471-2458-10-741

[24]   Maini, R., Mounier-Jack, S. and Borghi, J. (2017) How to and How Not to Develop a Theory of Change to Evaluate a Complex Intervention: Reflections on an Experience in the Democratic Republic of Congo. BMJ Global Health, 3, e000617.
https://doi.org/10.1136/bmjgh-2017-000617

[25]   Piebalgs, A. (2010) Global Health: A Millennium Development Goal and a Right for All, EU Commissioner for Development, at the Delivering the Right to Health with the Health MDGs Conference, Brussels.

[26]   Public Health Ministry (2012) DRC: MDG 4, 5 Acceleration and Achievement Framework.

[27]   Public Health Ministry (2015) DRC: National Plan for Health Development 2016-2020.

[28]   Public Health Ministry (2017) DRC, UNICEF, SANRU—Global Fund: Final Report of the Review of Community Case Management and Exchange of Experiences between Countries.

[29]   Boone, P., Elbourne, D., Fernands, S., et al. (2016) Effects of Community Health Interventions on Under-5 Mortality in Rural Guinea—Bissau (EPICS): A Cluster-Randomised Controlled Trial. Lancet Global Health, 4, e328-e335.
https://doi.org/10.1016/S2214-109X(16)30048-1

[30]   Arifeen, S.E., Hoque, E., Akter, T., et al. (2009) Effect of the Integrated Management of Childhood Illness Strategy on Childhood Mortality and Nutrition in a Rural Area in Bangladesh: A Cluster-Randomised Trial. The Lancet, 374, 393-403.

[31]   Gera, T., Garner, P., Richardson, M., et al. (2016) Integrated Management of Childhood Illness (IMCI) Strategy for Children under Five. Cochrane Database of Systematic Reviews, No. 6, CD010123.
https://doi.org/10.1002/14651858.CD010123.pub2

[32]   Marchal, B., Dedzo, M. and Kegels, G. (2010) A Realist Evaluation of the Management of a Well-Performing Regional Hospital in Ghana. BMC Health Services Research, 10, 24.
https://doi.org/10.1186/1472-6963-10-24

[33]   African Strategies for Health (ASH) (2016) Demand-Side Determinants to Integrated Community Case Management Care-Seeking. Lessons from the Democratic Republic of Congo.

[34]   DFID, UNICEF, World Bank, USAID and WHO (2016) The Analytic Review of Integrated Management of Childhood Illness Strategy. Final Report, World Health Organization, Geneva.

[35]   WHO (2011) Monitoring, Evaluation and Review of National Health Strategies: Country Platform for Information and Accountability. Geneva.

[36]   Bryce, J., Victora, C.G. and MCE-IMCI Technical Advisors (2005) Ten Methodological Lessons from the Multi-Country Evaluation of Integrated Management of Childhood Illness. Health Policy Plan, 20, i94-i105.

 
 
Top