Background: Crowding is a major challenge faced by EDs and is associated with poor outcomes.

Objectives: Determine the effect of high ED occupancy on disposition decisions, return ED visits, and hospitalizations.

Methods: We conducted a retrospective analysis of electronic health records of patients evaluated at an adult, urban, and academic ED over 20 months between the years 2012 and 2014. Using a logistic regression model predicting admission, we obtained estimates of the effect of high occupancy on admission disposition, adjusted for key covariates. We then stratified the analysis based on the presence or absence of high boarder patient counts.

Results: Disposition decisions during a high occupancy hour decreased the odds of admission (OR = 0.93, 95% CI: [0.89, 0.98]). Among those who were not admitted, high occupancy was not associated with increased odds of return in the combined (OR = 0.94, 95% CI: [0.87, 1.02]), with-boarders (OR = 0.96, 95% CI: [0.86, 1.09]), and no-boarders samples (OR = 0.93, 95% CI: [0.83, 1.04]). Among those who were not admitted and who did return within 14 days, disposition during a high occupancy hour on the initial ED visit was not associated with a significant increased odds of hospitalization in the combined (OR = 1.04, 95% CI: [0.87, 1.24]), the with-boarders (OR = 1.12, 95% CI: [0.87, 1.44]), and the no-boarders samples (OR = 0.98, 95% CI: [0.77, 1.24]).

Conclusion: ED crowding was associated with reduced likelihood of hospitalization without increased likelihood of 2-week return ED visit or hospitalization. Furthermore, high occupancy disposition hours with high boarder patient counts were associated with decreased likelihood of hospitalization.

Independent verification is a critical component of performance-based financing (PBF) in health care, in which facilities are offered incentives to increase the volume of specific services but the same incentives may lead them to over-report. We examine alternative strategies for targeted sampling of health clinics for independent verification. Specifically, we empirically compare several methods of random sampling and predictive modeling on data from a Zambian PBF pilot that contains reported and verified performance for quantity indicators of 140 clinics. Our results indicate that machine learning methods, particularly Random Forest, outperform other approaches and can increase the cost-effectiveness of verification activities.

Background: Performance-based financing (PBF) both measures and determines payments based on the quality of care delivered and is emerging as a potential tool to improve quality.

Methods: Comparative case study methodology was used to analyze common challenges and lessons learned in quality of care across seven PBF programs (Democratic Republic of Congo, Kyrgyzstan, Malawi, Mozambique, Nigeria, Senegal and Zambia). The eight case studies, across seven PBF programs, compared were commissioned by the USAID-funded Translating Research into Action (TRAction) project (n = 4), USAID’s Health Finance and Government project (n = 3), and from the Global Delivery Initiative (n = 1).

Results: The programs show similar design features to assess quality, but significant heterogeneity in their application. The seven programs included 18 unique quality checklists, containing over 1400 quality of care indicators, with an average per checklist of 116 indicators (ranging from 26-228). The quality checklists share a focus on structural components of quality (representing 80% of indicators on average, ranging from 38%-91%). Process indicators constituted an average of 20% across all checklists (ranging from 8.4% to 61.5%), with the majority measuring the correct application of care protocols for MCH services including child immunization. The sample included only one example of an outcome indicator from Kyrgyzstan. Performance data demonstrated a modest upward improvement over time in checklist scores across schemes, however, achievements plateaued at 60%-70%, with small or rural clinics reporting difficulty achieving payment thresholds due to limited resources and poor infrastructure. Payment allocations (distribution) and thresholds (for payments), data transparency, and approaches to measuring (verification) of quality differ across schemes.

Conclusions: Similarities exist in the processes that govern the design of PBF mechanisms, yet substantial heterogeneity in the experiences of implementing quality of care components in PBF programs are evident. This comparison suggests tailoring further the quality component of PBF programs to local and country contexts, and a need to better understand how quality is measured in practice. The growing operational experiences with PBF programs in different settings offer opportunities to learn from best practices, improve ongoing and future programs, and inform research to alleviate current challenges.

Governments across low-income and middle-income countries have pledged to achieve universal health coverage by 2030, which comes at a time where healthcare systems are subjected to multiple and persistent pressures, such as poor access to care services and insufficient medical supplies. While the political willingness to provide universal health coverage is a step into the right direction, the benefits of it will depend on the quality of healthcare services provided. In this analysis paper, we ask whether there are any lessons that could be learnt from the English National Health Service, a healthcare system that has been providing comprehensive and high-quality universal health coverage for over 70 years. The key areas identified relate to the development of a coherent strategy to improve quality, to boost public health as a measure to reduce disease burden, to adopt evidence-based priority setting methods that ensure efficient spending of financial resources, to introduce an independent way of inspecting and regulating providers, and to allow for task-shifting, specifically in regions where staff retention is low.

Summary box

• The evidence surrounding the cost-effectiveness of performance-based financing (PBF) is weak, and it is not clear how PBF compares with alternative interventions in terms of its value for money.

• It is important to fill this evidence gap as countries transition from aid and face increasing budget constraints and competing priorities for the use of their domestic resources.

• In conducting cost-effectiveness analyses of PBF, researchers should be mindful of the identification, measurement and valuation of costs and effects, provide justification for the scope of their studies, and specify appropriate comparators and decision rules.

• We also recommend the use of a reference case to lay out the principles, preferred methodological choices and reporting standards, as well as a checklist.

In the wake of the European refugee crisis, Germany has received over a million new applications for asylum in the last two years. The health care system is struggling to provide asylum-seekers with access to essential medical services and facilitate their longer-term integration. In this article, we report on the morbidity, utilization and costs of care for a sample of asylum-seekers as compared to a matched group of regularly insured. Using administrative data, we found that asylum-seekers had more hospital and emergency department admissions, including more admissions that could be avoided through good outpatient care or prevention. Their average expenditures were 10 percent higher than for the regularly insured, mostly because of higher hospital expenditures, although there was substantial variation in expenditures by country of origin. Facilitating access to the health care system, especially outpatient and mental health care, could improve asylum-seekers health status and integration, possibly at lower costs.

Introduction: Citizen report cards on health care providers have been identified as a potential means to increase citizen engagement, provider accountability and health systems performance.  Research in high-income settings indicates that the wording, presentation and display of performance information are critical to achieve these goals. However, there are limited insights on developing effective report card designs for middle- and low-income settings.  We conducted cognitive interviews to assess consumers’ understanding, interpretation of and preferences for displaying information for a health care report card in rural Tajikistan.

Materials and Methods: We recruited a convenience sample of 40 citizens (20 women and 20 men aged 18-45) from rural areas of two provinces of Tajikistan (Soghd and Khatlon oblasts).  The interview protocol was adapted from the model of cognitive interviews used in social science research to improve survey questionnaires.  We used multivariate regression to assess understanding and interpretation of the report card; chi2 tests to assess differences in preferences for displaying information; and tests of proportions to assess the preferred comparison group.

Results: Respondents understood the main idea of the report card and are not confused by the indicators or display.  However, many respondents had difficulties making comparisons, and when asked to identify worst-performing services.  Respondents preferred detailed rankings using school grades, comparisons of their local clinic with the regional or national average performance, and the use of color in the report card.  We found some heterogeneity across the two provinces.

Conclusions: Overall, our findings are promising regarding the citizens’ comprehension of health care report cards in rural Tajikistan, while underscoring the challenges of effectively providing health care performance information to communities.  Cognitive interviews and iterative testing can support an effective implementation of reporting initiatives.

Objective: To systematically describe the length and content of quality checklists used in performance-based financing programs, their similarities and differences, and how checklists have evolved over time.

Methods: We compiled a list of supply-side, health facility-based performance-based financing (PBF) programs in low- and lower middle-income countries based on a document review. We then solicited PBF manuals and quality checklists from implementers and donors of these PBF mechanisms. We entered each indicator from each quality checklist into a database verbatim in English, and translated into English from French where appropriate, and categorized each indicator according to the Donabedian framework and an author-derived categorization.

Findings: We extracted 8,490 quality indicators from 68 quality checklists across 32 PBF implementations in 28 countries. On average, checklists contained 125 indicators; within the same program, checklists tend to grow as they are updated. Using the Donabedian framework, 80% of indicators were structure-type, 19% process-type, and less than 1% outcome-type. The author-derived categorization showed that 57% of indicators relate to availability of resources, 24% to managing the facility and 17% assess knowledge and effort. There is a high degree of similarity in a narrow set of indicators used in checklists for common service types such as maternal, neonatal and child health.

Conclusion:  Performance-based financing offers an appealing approach to targeting specific quality shortfalls and advancing toward the Sustainable Development Goals of high quality coverage.  Currently most indicators focus on structural issues and resource availability. There is scope to rationalize and evolve the quality checklists of these programs to help achieve national and global goals to improve quality of care.

Many competitive health insurance markets adjust payments to participating health plans according to their enrollees’ risk – including based on diagnostic information.  We investigate responses of German health plans to the introduction of morbidity-based risk adjustment in the Statutory Health Insurance in 2009, which triggers payments based on “validated” diagnoses by providers.  Using the regulator’s data from office-based physicians, we estimate a difference-in-difference analysis of the change in the share and number of validated diagnoses for ICD codes that are inside or outside the risk adjustment but are otherwise similar.  We find a differential increase in the share of validated diagnoses of 2.6 and 3.6 percentage points (3-4%) between 2008 and 2013.  This increase appears to originate from both a shift from not-validated toward validated diagnoses and an increase in the number of such diagnoses.  Overall, our results indicate that plans were successful in influencing physicians’ coding practices in a way that could lead to higher payments.

Also available as CESIfo Working Paper 6507

Objective: To describe how quality of care is incorporated into performance-based financing (PBF) programs, what quality indicators are being used, and how these indicators are measured and verified.

Methods: An exploratory scoping methodology was used to characterize the full range of quality components in 32 PBF programs, initiated between 2008 and 2015 in 28 low- and middle-income countries, totaling 68 quality tools and 8,490 quality indicators. The programs were identified through a review of the peer-reviewed and gray literature as well as through expert consultation with key donor representatives.

Findings: Most of the PBF programs were implemented in sub-Saharan Africa and most were funded primarily by the World Bank. On average, PBF quality tools contained 125 indicators predominately assessing maternal, newborn, and child health and facility management and infrastructure. Indicators were primarily measured via checklists (78%, or 6,656 of 8,490 indicators), which largely (over 90%) measured structural aspects of quality, such as equipment, beds, and infrastructure. Of the most common indicators across checklists, 74% measured structural aspects and 24% measured processes of clinical care. The quality portion of the payment formulas were in the form of bonuses (59%), penalties (27%), or both (hybrid) (14%). The median percentage (of a performance payment) allocated to health facilities was 60%, ranging from 10% to 100%, while the median percentage allocated to health care providers was 55%, ranging from 20% to 80%. Nearly all of the programs included in the analysis (91%, n=29) verified quality scores quarterly (every 3 months), typically by regional government teams.

Conclusion: PBF is a potentially appealing instrument to address shortfalls in quality of care by linking verified performance measurement with strategic incentives and could ultimately help meet policy priorities at the country and global levels, including the ambitious Sustainable Development Goals. The substantial variation and complexity in how PBF programs incorporate quality of care considerations suggests a need to further examine whether differences in design are associated with differential program impacts.

Background: The Affordable Care Act established policy mechanisms to increase health insurance coverage in the United States. While insurance coverage has increased, 10%-15% of the US population remains uninsured.

Objectives: To assess whether health insurance literacy and financial literacy predict being uninsured, covered by Medicaid, or covered by Marketplace insurance, holding demographic characteristics, attitudes toward risk, and political affiliation constant.

Research Design: Analysis of longitudinal data from fall 2013 and spring 2015 including financial and health insurance literacy and key covariates collected in 2013.

Subjects: A total of 2742 US residents ages 18-64, 525 uninsured in fall 2013, participating in the RAND American Life Panel, a nationally representative internet panel.

Measures: Self-reported health insurance status and type as of spring 2015.

Results: Among the uninsured in 2013, higher financial and health insurance literacy were associated with greater probability of being insured in 2015. For a typical uninsured individual in 2013, the probability of being insured in 2015 was 8.3 percentage points higher with high compared with low financial literacy, and 9.2 percentage points higher with high compared with low health insurance literacy. For the general population, those with high financial and health insurance literacy were more likely to obtain insurance through Medicaid or the Marketplaces compared with being uninsured. The magnitude of coefficients for these predictors was similar to that of commonly used demographic covariates.

Conclusions: A lack of understanding about health insurance concepts and financial illiteracy predict who remains uninsured. Outreach and consumer-education programs should consider these characteristics.

Pay-for-performance (P4P) programs have been introduced in numerous developing countries with the goal of increasing the provision and quality of health services through financial incentives. Despite the popularity of P4P, there is limited evidence on how providers achieve performance gains and how P4P affects health system quality by changing structural inputs. We explore these two questions in the context of Rwanda's 2006 national P4P program by examining the program's impact on structural quality measures drawn from international and national guidelines. Given the program's previously documented success at increasing institutional delivery rates, we focus on a set of delivery-specific and more general structural inputs. Using the program's quasi-randomized rollout, we apply multivariate regression analysis to short-run facility data from the 2007 Service Provision Assessment. We find positive program effects on the presence of maternity-related staff, the presence of covered waiting areas, and a management indicator and a negative program effect on delivery statistics monitoring. We find no effects on a set of other delivery-specific physical resources, delivery-specific human resources, delivery-specific operations, general physical resources, and general human resources. Using mediation analysis, we find that the positive input differences explain a small and insignificant fraction of P4P's impact on institutional delivery rates. The results suggest that P4P increases provider availability and facility operations but is only weakly linked with short-run structural health system improvements overall.

Background: Given plans to extend its regulatory authority to e-cigarettes, the Food and Drug Administration (FDA) urgently needs to understand how e-cigarettes are perceived by the public.

Objectives: To examine how smoking status impacts adult perceptions and expectations of e-cigarettes.

Methods: We used Mechanical Turk (MTurk), a “crowdsourcing” platform, to rapidly survey a large (n=796; female=381; male=415), diverse sample of adult ever (44%) and never smokers (56%), with ever (28%) and never (72%) users of e-cigarettes.  

Results: Smokers and non-smokers learned about e-cigarettes primarily through the internet and conversations with others.  Ever smokers were more likely than never smokers, and female current smokers were more likely than female former smokers, to have learned about e-cigarettes from point of sale advertising (p’s<0.05) and to believe that e-cigarettes help smokers quit (p’s<0.05).  Among never-users of e-cigarettes, current smokers were more likely than never smokers and former smokers to report that they would try e-cigarettes in the future (p’s<0.01). Current smokers’ top reason for wanting to try e-cigarettes was to quit or reduce smoking (56%), while never and former smokers listed curiosity. In contrast, female current smokers’ top reason for not trying e-cigarettes was health and safety concerns (44%) while males indicated expense (44%).

Conclusions: Adult smokers and non-smokers have different perceptions and expectations of e-cigarettes.  Public health messages regarding e-cigarettes may need to be tailored separately for persons with and without a history of using conventional cigarettes. Tailoring messages by gender within smoker groups may also improve their impact.

Performance-based contracting is particularly challenging in health care, where multiple agents, information asymmetries and other market failures compound the critical contracting concern of multitasking. As performance-based contracting grows in developing countries, it is critical to better understand not only intended program impacts on rewarded outcomes, but also unintended program impacts such as multitasking and heterogeneous program effects in order to guide program design and scale-up. We use two waves of data from the Rwanda Demographic and Health Surveys collected before and after the quasi-experimental roll- out of Rwanda’s national pay-for-performance (P4P) program to analyze impacts on utilization of healthcare services, health outcomes and unintended consequences of P4P. We find that P4P improved some rewarded services, as well as some services that were not directly rewarded, but had no statistically significant impact on health outcomes. We do not find evidence that clearly suggests multitasking. We find that program effects vary by baseline levels of facility quality, with most improvements seen in the medium quality tier.

Transparency interventions, such as public reporting, have emerged as a potential policy approach to improving the performance of health care providers in resource-constrained settings.  We report on results from focus groups and key informant interviews in rural areas of two Tajik provinces, Soghd and Khatlon, with regards to three important initial considerations for developing a report card initiative for primary health care in this setting: selecting indicators for the report card, collecting data, and working with existing institutions and stakeholders.  The findings suggest that citizens are able to articulate and prioritize concerns with respect to local health care services.  Participants indicated a preference for arms-length collection of sensitive feedback on local providers.  Since citizens and local institutions have close and important relations with their local health care providers, there may be scope for a trusted external actor, such as a non-governmental organization, to facilitate the report card process.

The routine data generated by India’s universal coverage programs offer an important opportunity to evaluate and track the quality of health care systematically and on a large scale. We examined the potential and challenges of measuring the quality of hospital care through claims data from India’s hospital insurance program for the poor, Rashtriya Swasthya Bima Yojana (RSBY). Using data from one district in India, we illustrate how these data already provide useful insights and show that simple efforts to enhance data quality and an effort to expand the data captured could facilitate RSBY’s ability to track quality of care. The data collected by RSBY has significant potential to characterize and uncover the provision of low-quality care and help inform much-needed efforts to raise the quality of hospital care.

Quality of care is emerging as an important concern for low- and middle-income countries working to expand and improve coverage. However, there is limited systematic, large-scale empirical guidance to inform policy design. Our study operationalized indicators for six dimensions of quality of care that are captured in currently available, standardized Service Provision Assessments. We implemented these measures to assess the levels and heterogeneity of antenatal care in Kenya. Using our indicator mix, we find that performance is low overall and that there is substantial variation across provinces, management authority and facility type. Overall, facilities performed highest in the dimensions of efficiency and acceptability/patient-centeredness, and lowest on effectiveness and accessibility. Public facilities generally performed worse or similarly to private or faith-based facilities. We illustrate how these data and methods can provide readily-available, low-cost decision support for policy.