Research

Working Paper
Jonathan Roth. Working Paper. “Pre-test with Caution: Event-study Estimates After Testing for Parallel Trends”.Abstract
Researchers using an event-study design often test for pre-event trends (``pre-trends''), yet typical estimation and inference does not account for this test. This paper analyzes the properties of conventional event-study estimates conditional on having survived a test for pre-trends. Pre-testing for pre-trends changes the bias of conventional estimates when parallel trends is violated. I show that in settings with homoskedastic errors and a monotone trend, the bias conditional on surviving a pre-test is larger than the unconditional bias. Hence, pre-trends tests meant to mitigate bias can actually exacerbate it. Pre-testing also distorts the coverage rates of conventional confidence intervals, which can be above or below their nominal level conditional on surviving the pre-test. Simulations based on a systematic review of recent papers in leading economics journals suggest that conventional pre-tests are often underpowered and substantial distortions from pre-testing are possible in practice. To address these issues, I develop a method to correct event-study plots for the distortions from pre-testing. I recommend that researchers who rely on pre-trends testing report these corrected event-studies along with calculations of the power of the pre-test against economically relevant alternatives.
roth_pretrends_20190730.pdf
Ashesh Rambachan and Jonathan Roth. Working Paper. “Bias In, Bias Out? Evaluating the Folk Wisdom”.Abstract
We evaluate the folk wisdom that algorithms trained on data produced by biased human decision-makers necessarily reflect this bias. We consider a setting where training labels are only generated if a biased decision-maker takes a particular action, and so bias arises due to selection into the training data. In our baseline model, the more biased the decision-maker is toward a group, the more the algorithm favors that group. We refer to this phenomenon as "algorithmic affirmative action." We then clarify the conditions that give rise to algorithmic affirmative action. Whether a prediction algorithm reverses or inherits bias depends critically on how the decision-maker affects the training data as well as the label used in training. We illustrate our main theoretical results in a simulation study applied to the New York City Stop, Question and Frisk dataset.
Paper.pdf
Isaiah Andrews, Jonathan Roth, and Ariel Pakes. Working Paper. “Inference for Linear Conditional Moment Inequalities”.Abstract
We consider inference based on linear conditional moment inequalities, which arise in a wide variety of economic applications, including many structural models.  We show that linear conditional structure greatly simplifies confidence set construction, allowing for computationally tractable projection inference in settings with nuisance parameters.  Next, we derive least favorable critical values that avoid conservativeness due to projection.  Finally, we introduce a conditional inference approach which ensures a strong form of insensitivity to slack moments, as well as a hybrid technique which combines the least favorable and conditional methods.  Our conditional and hybrid approaches are new even in settings without nuisance parameters.  We find good performance in simulations based on Wollmann (2018), especially for the hybrid approach.
Paper.pdf Supplement.pdf
Jonathan Roth. Working Paper. “Union Reform and Teacher Turnover: Evidence from Wisconsin's Act 10”.Abstract
This paper studies teacher attrition in Wisconsin following Act 10, a policy change which severely weakened teachers’ unions and capped wage growth for teachers. I document a sharp short-run increase in teacher turnover after the Act was passed, driven almost entirely by teachers over the minimum retirement age of 55, whose turnover rate doubled from 17 to 35 percent. Such teachers faced strong incentives to retire before the end of pre-existing collective bargaining agreements in order to secure collectively-bargained retirement benefits (e.g. healthcare), which no longer fell under the scope of collective bargaining after the Act. I find much more modest long-run increases in teacher turnover, consistent with previous estimates of labor supply elasticities. I then attempt to evaluate the effect of the wave of retirements following Act 10 on education quality using grade-level value-added metrics. I find suggestive evidence that student academic performance increased in grades with teachers who retired following the reform, and I obtain similar results when instrumenting for retirement using the pre-existing age distribution of teachers. Differences in value-added between retirees and their replacements can potentially explain some, but not all, of the observed academic improvements.
roth_act10_20190329.pdf