.
. Chevy Chase, MD: USAID Applying Science to Strengthen and Improve Systems (ASSIST) Project/URC, 2017.
EXECUTIVE SUMMARY
Background
Institutionalization and sustainability are important issues for any donor funded program as the effectiveness of the initiatives are usually expected to persist after the donor funding period has ended. This report develops a framework for analysis integrating the two concepts and applies it to an important case of donor-funded quality improvement projects in Honduras whose funding terminated in 2012.
Honduras is a lower-middle-income country in Central America. The population is predominantly young, with fertility and under-five mortality rates similar to those in other countries in the region. Despite low levels of health care spending, the country had achieved substantial reductions in mortality rates at different ages.
The Quality Assurance Project (QAP), funded by the United States Agency for International Development and implemented by University Research Co., LLC (URC), began providing technical assistance to the Honduras Secretariat of Health (SSH) in 1997, designing and implementing a quality assurance (QA) system to improve the quality of maternal and child health services in Health Region 2 (Comayagua), including standards development, compliance monitoring, quality design, and quality improvement (QI) teams. ln 2004, SSH changed its organizational structure from eight health regions to 20 departmental regions and requested that QAP support the scale-up of QI in the five USAID-assisted departmental regions: Copán, Comayagua, La Paz, Intibucá, and Lempira. QAP also supported the local implementation of the national QA regulatory mechanism of facility licensing and later participation of facility teams in a regional essential obstetric and newborn care improvement collaborative.
QAP’s assistance in Honduras was directly followed by support from the USAID Health Care Improvement Project (HCI) from 2007 to 2012. HCI supported the SSH in the implementation of the National Plan for Maternal and Child Mortality Reduction, and during 2010-2011, provided technical assistance to develop a National Quality Health System.
Methods
This case study was designed to ask how much of the projects’ results, funding, policies, and culture continued five years after the resources and technical assistance were withdrawn and at what level of fidelity, and what factors could explain the sustainability or lack of sustainability.
The information to inform the analysis was collected through documents, statistics, and interviews. The analysis of documents included the projects’ description and reports. Interviewees were selected in order to represent different actors involved in the design and implementation of the QAP and HCI projects, following a judgment-based sample. Twelve interviews were conducted in November and December 2016: four at the national level, including national health authorities and civil servants at SSH; five of program managers and implementers at health facilities; and three of representatives of international agencies working in quality improvement of maternal and child health in Honduras. All the interviews were conducted in Spanish, in person.
Findings
The availability of resources and technical support from international institutions facilitated the implementation of the quality initiatives in the country. The Secretariat of Health led the project and it was “institutionalized” at national level through the Quality Management System, designed and implemented by the SSH Quality Management Unit. Although most of the activities of the project were carried out at the local level--mostly training and technical assistance to develop and implement facility-level improvement plans--the project’s goals were embraced by authorities at the central level. Political support for the project and commitment with quality improvement in healthcare gave rise to the creation in 2011 of the National Policy for Quality in Health Care. “When URC ended the project, the torch was taken up by the Institutionalization of quality assurance initiatives in Honduras Unit of Quality Management at the SSH… The Unit’s technical profile was adequate to continue doing many of the activities of the project. This gave continuity to the process”, said a national coordinator.
Regarding sustainability, most stakeholders acknowledge that many activities that started with the QAP and HCI projects are still in place: “I think many things continued. Even with new elements, without the influence of URC… I think the main impact (after the project withdrew) is that we stopped moving forward”, said a stakeholder at the SSH. Interviewees mentioned visible products, such as quality units and the National Quality Congress. Although they all agreed that these activities and organizations still exist, some doubts were raised about their functionality. One of the critical aspects identified by several interviewees is the lack of trained human resources to continue the project’s activities. Interviews highlighted that central-level personnel trained during the duration of the project have continued working on promoting quality, but many of them are no longer working at the SSH or are close to retirement.
At local level, quality improvement projects were well understood: “In general, people understood the projects (continuous improvement), I will say in 85%... Standard, indicators, measures. People knew these things… They know the methodology.” Other factors previously highlighted refer to the flexibility of the intervention, and its ability to be adapted by local teams. On the other hand, technical requirements were also mentioned as one potential barrier to institutionalization and continuation of the activities of the projects. Several stakeholders flagged the lack of monitoring and evaluation systems as critical in explaining the stagnation of processes initiated by URC.
Conclusions
The projects implemented by URC in the country for more than a decade set the basis to maintain a process of continuous improvement in Honduras. In terms of results, interviewees agree that the main component of the project was the quality training. This element helped to create a culture of quality in health care, giving policy makers and health professionals tools to work on the priority issue of quality improvement. The project also generated important administrative changes, such as the creation of quality units at the central and local levels.
Interviewees recognized that there are still many improvements required to produce impacts in maternal and child mortality in the country and that quality itself is a never-ending process. This is a key element of the project’s approach that improvement is a continual process of adaptation. However, there was a repeated theme that despite the successes of the projects during their implementation, there has been a fall-off in terms of sustaining the same level of activity as during the projects. Despite the project’s orientation that emphasizes the need to move on from achieving improvements in one quality area to improve other quality problems, it is clear that after HCI ended, teams moving on to other quality issues did not occur. The activities that were achieved during the life of the project were partly sustained but not at the same level as during the life of the project.
Interviewees attributed this deterioration to the lack of dedicated resources and technical guidance for QI, turning important QI processes into mere formalities. There was a failure to monitor and follow up and there were insufficient incentives to overcome the tendency for staff to prioritize other activities. In addition, the staff which had been trained during the project moved on to other activities or retired and there was little attention to training of new staff.
Some major lessons for future projects are that project design should take into account the multiple factors involved in the institutionalization of project outcomes. In Honduras, strong attention to developing effective organizations like the committees at the national and local levels was important and instilled a strong culture around the objectives and activities of the project. The Honduras experience also shows how important it is to develop ongoing financial support from multiple sources (perhaps including other external funders), to establish ongoing training programs to replace the trained staff who retire or move on to other activities, and to develop incentives and commitment to maintaining a high level of quality monitoring.