In this paper, we develop bias formulas for front-door estimates and front-door/back- door hybrid estimates of average treatment effects under general patterns of measured and unmeasured confounding. These bias formulas allow for sensitivity analysis, and also allow for comparisons of the bias resulting from standard back-door covariate ad- justments (also known as direct adjustment and standardization). We also present these bias comparisons in two special cases: linear structural equation models and nonrandomized program evaluations with one-sided noncompliance. These compar- isons demonstrate that there are broad classes of applications for which the front-door or hybrid adjustments will be preferred to the back-door adjustments. We illustrate this point with an application to the National JTPA (Job Training Partnership Act) Study, showing that by using information on enrollment in addition to pre-treatment covariates, the front-door approach provides estimates that are closer to the experi- mental benchmark than the back-door approach.
Using the Rosenbaum (2002, 2009) approach to observational studies, we show how qualitative information can be incorporated into quantitative analyses to improve causal inference in three ways. First, we can ameliorate the effects of difficult-to-measure outcomes by including qualitative information on outcomes within matched sets, sometimes reducing p-values. Second, additional information across matched sets enables the construction of qualitative confidence intervals on effect size. Third, qualitative information on unmeasured confounders within matched sets reduces the conservativeness of Rosenbaum-style sensitivity analysis. This approach accommodates small to medium sample sizes in a nonparametric framework, and therefore may be particularly useful for analyses of the effects of institutions in a given set of countries or subnational units. We illustrate these methods by examining the effect of using plurality rules in transitional presidential elections on opposition harassment in 1990s sub-Saharan Africa.
In many situations data are available at the group
level but one wishes to estimate the individual-level association
between a response and an explanatory variable. Unfortunately this
endeavor is fraught with difficulties because of the ecological level
of the data. The only reliable solution to such ecological inference
problems is to supplement the ecological data with individual-level
data. In this paper we illustrate the benefits of gathering
individual-level data in the context of a Poisson modeling
framework. Additionally, we derive optimal designs that allow the
individual samples to be chosen so that information is maximized. The
methods are illustrated using Robinson's classic data on illiteracy rates. We show that the optimal design produces accurate
inference with respect to estimation of relative risks, with
ecological bias removed.
In this paper, we leverage the natural experiment of a child’s gender to identify the effect of having daughters on the votes of judges. Using new data on the family lives of U.S. Courts of Appeals judges, we find that, conditional on the number of children a judge has, judges with daughters consistently vote in a more pro-woman fashion on gender issues than judges who have only sons. This result survives a number of robustness tests and appears to be driven primarily by Republican judges. More broadly, this result demonstrates that personal experiences influence how judges make decisions, and it is the first paper to show that empathy may indeed be a component in how judges decide cases.
We explicitly delineate the underlying homogeneity assumption, procedural variants, and implications of the comparative method [Lijphart, 1975] and distinguish this from Mill’s method of difference . We demonstrate that additional units can provide “placebo” tests for the comparative method even if the scope of inference is limited to the two units under comparison. Moreover, such tests may be available even when these units are the most similar pair of units on the control variables with differing values of the independent variable. Small-n analyses using this method should therefore, at a minimum, clearly define the dependent, independent, and control variables so they may be measured for additional units, and specify how the control variables are weighted in defining similarity between units. When these tasks are too difficult, process tracing of a single unit may be a more appropriate method. We illustrate these points with applications to Epstein  and Moore .
Research design is of paramount importance when attempting to overcome con- founding. In this paper, we propose a unified graphical approach for the consideration of cross-sectional research designs. Specifically, we argue that at least five distinct strategies may be discerned for coping with the presence of a common-cause con- founder: (1) blocking backdoor paths, (2) mechanisms, (3) instrumental variables, (4) alternate outcomes, and (5) causal heterogeneity. All of these strategies enlist a facil- itating variable, whose role defines the corresponding research design. This resulting framework builds on the foundational work of Pearl (2000, 2009) but incorporates addi- tional research designs into the graphical framework, providing a more comprehensive typology of designs for causal inference.
The structural causal models (SCM) of Pearl (1995, 2000, 2009) provide a graphical criterion for choosing the “right hand side” variables to include in a model. In this paper, we use SCMs to address the question of whether to include lagged variables in time-series-cross-section (TSCS) models. This question has received a great deal of attention from political methodologists, but unfortunately, the practical advice for applied researchers that comes out of this literature varies considerably from article to article. We attempt to clarify the nature of some of these disagreements and to provide useful tools to reason about the nonparametric identification of causal effects. After clarifying the debate between Beck and Katz (1996, 2011) and Achen (2000) and adding to the discussion by Keele and Kelly (2006), we provide concrete nonparametric identification results for commonly studied TSCS data generating processes. These results are also relevant for the choice of control variables in cross-section (CS) models. We conclude with some general thoughts on how a focus on using the SCM as a tool for proving identification results can help TSCS and CS researchers do better work.
In this paper, we illustrate that combining ecological data with subsample data in situations in which a generalized linear model (GLM) is appropriate provides two main benefits. First, by including the individual level subsample data, the biases associated with ecological inference in GLMs can be eliminated. Second, available ecological data can be used to design optimal subsampling schemes, so as to maximize information about parameters. We present an application of this methodology to voter turnout studies showing that small, optimally chosen subsamples can be combined with ecological data to generate precise estimates relative to a simple random subsample, and we discuss possible applications in epidemiology.
Conflict scholars have devoted considerable attention to the natural resource curse, and specifically to connections between natural resources, state weakness, and civil war. Many have posited a state weakness mechanism-- that significant oil production causes state weakness, and state weakness consequently increases the likelihood of civil war onset. Using standard measures, this paper demonstrates that the state weakness mechanism does not exist in the short or medium term. The methods developed in this paper show that in only two cases is there the possibility of a medium term effect, and the state weakness mechanism is unlikely to be operative even in these two cases. Furthermore, these methods do not rely on assumptions about unmeasured confounders, so this result is robust to the consideration of other risk factors for civil war onset. The state weakness mechanism may still exist in the form of long term effects or an effect that reinforces pre-existing war and/or state weakness. However, the null hypothesis of no long-term and/or reinforcing effect cannot be rejected without the use of additional assumptions.
Due to the inherent sensitivity of many survey questions, a number of researchers have adopted an indirect questioning technique known as the list experiment (or the item count technique) in order to minimize bias due to dishonest or evasive responses. However, standard practice with the list experiment requires a large sample size, is not readily adaptable to regression or multivariate modeling, and provides only limited diagnostics. This paper addresses all three of these issues. First, the paper presents design principles for the standard list experiment (and the double list experiment) to minimize bias and reduce variance as well as providing sample size formulas for the planning of studies. Additionally, this paper investigates the properties of a number of estimators and introduces an easy-to-use piecewise estimator that reduces necessary sample sizes in many cases. Second, this paper proves that standard-procedure list experiment data can be used to estimate the probability that an individual holds the socially undesirable opinion/behavior. This allows multivariate modeling. Third, this paper demonstrates that some violations of the behavioral assumptions implicit in the technique can be diagnosed with the list experiment data. The techniques in this paper are illustrated with examples from American politics.
Political scientists often cite the importance of mechanism-specific causal knowledge, both for its intrinsic scientific value and as a necessity for informed policy. This article explains why two common inferential heuristics for mechanism-specific (i.e., indirect) effects can provide misleading answers, such as sign reversals and false null results, even when linear regressions provide unbiased estimates of constituent effects. Additionally, this article demonstrates that the inferential difficulties associated with indirect effects can be ameliorated with the use of stratification, interaction terms, and the restriction of inference to subpopulations (e.g., the indirect effect on the treated). However, indirect effects are inherently not identifiable— even when randomized experiments are possible. The methodological discussion is illustrated using a study on the indirect effect of Islamic religious tradition on democracy scores (due to the subordination of women).
Our goal in this paper is to provide a formal explanation for how within-unit causal process information (i.e., data on posttreatment variables and partial information on posttreatment counterfactuals) can help to in- form causal inferences relating to total effects—the overall effect of an explanatory variable on an outcome variable. The basic idea is that, in many applications, researchers may be able to make more plausible causal assumptions conditional on the value of a posttreatment variable than they would be able to do unconditionally. As data become available on a posttreatment variable, these conditional causal assumptions become active and information about the effect of interest is gained. This approach is most beneficial in situations where it is implausible to assume that treatment assignment is conditionally ignorable. We illustrate the approach with an example of estimating the effect of election day registration on turnout.
Ecological inference is a problem of partial identification, and
therefore precise conclusions are rarely possible without
the collection of individual level (identifying) data. Without such
data, sensitivity analyses provide the only recourse. In this paper
we review and critique recent approaches to ecological inference in the
social sciences, and describe in detail hierarchical models, which
allow both sensitivity analysis and the incorporation of individual
level data into an ecological analysis. A crucial element of a
sensitivity analysis in such models is prior specification, and we
detail how this may be carried out. Furthermore, we demonstrate how
the inclusion of a small amount of individual level data from a small number of ecological areas can
dramatically improve the properties of such estimates.
In this paper we discuss an estimator for average treatment effects
known as the augmented inverse propensity weighted (AIPW). This
estimator has attractive theoretical properties and only requires
practitioners to do two things they are already comfortable with:
(1) specify a binary regression model for the propensity score, and
(2) specify a regression model for the outcome variable. After
explaining the AIPW estimator, we conduct a Monte Carlo experiment
that compares the performance of the AIPW estimator to three common
competitors: a regression estimator, an inverse propensity weighted
(IPW) estimator, and a propensity score matching estimator. The
Monte Carlo results show that the AIPW estimator is dramatically
superior to the other estimators in many situations and at least as
good as the other estimators across a wide range of data generating
In close elections, the losing side has an incentive to obtain evidence that the election result is incorrect. Sometimes this evidence comes in the form of court testimony from a sample of invalid voters, and this testimony is used to adjust vote totals (Belcher v. Mayor of Ann Arbor 1978; Borders v. King County 2005). However, while courts may be reluctant to make explicit findings about out-of-sample data (e.g., invalid voters that do not testify), when samples are used to adjust vote totals, the court is making such findings implicitly. In this paper, we show that the practice of adjusting vote totals on the basis of potentially unrepresentative samples can lead to incorrectly voided election results. More generally, we demonstrate that even when frame error and measurement error are minimal, random samples of post-vote vote-choice data can have limited power to detect incorrect election results without high response rates, precinct level polarization, or the acceptance of large Type I error rates. Therefore, in U.S. election disputes, even high-quality post-vote vote-choice data may be insufficient to resolve contested elections without the use of modeling assumptions (whether or not these assumptions are acknowledged).
In this paper, we illustrate that combining ecological data with subsample data in situations in which a linear model is appropriate provides two main benefits. First, by including the individual level subsample data, the biases associated with linear ecological inference can be eliminated. Second, we can use readily available ecological data to design optimal subsampling schemes, so as to maximize information about parameters. We present an application of this methodology to the classic problem of estimating the effect of a college degree on wages, showing that small, optimally chosen subsamples can be combined with ecological data to generate precise estimates relative to a simple random subsample.