Publications

Working Paper
Guryan, J., Kim, J. S., Capotosto, L., Quinn, D. M., Kingston, H. C., Foster, L., & Cooc, N. (Working Paper). Can a Scaffolded Summer Reading Intervention Reduce Socioeconomic Gaps in Children’s Reading Comprehension Ability and Home Book Access? Results from a Randomized Experiment. Northwestern University Institute for Policy Research Working PaperAbstract

The researchers conducted a randomized experiment involving 824 third-grade children in 14 elementary schools (K-5) to examine the effects of a scaffolded summer reading intervention that provided books matched to children’s reading level and interests and teacher scaffolding in the form of end-of-year comprehension instruction. Within each school, children and teachers were randomly assigned to (a) a control condition involving six math lessons, (b) a treatment condition with six reading comprehension lessons and an after school family literacy event, and (c) an enhanced treatment that also included follow-up phone calls to parents. During summer vacation, children in the treatment conditions received two lesson books and eight matched books. A treatment by socioeconomic status (SES) interaction effect on reading comprehension posttests revealed larger positive effects for children in high poverty schools than children in moderate-high poverty schools. In addition, among a random subsample of children (n = 121) who were part of a home visit study, there were positive treatment effects on the quantity and the diversity of books at home and trends suggested larger effects for children from low SES families. The results highlight the variability in treatment effects across different school and family contexts.

Guryan, J., Kim, J. S., & Quinn, D. M. (Working Paper). Does reading during the summer build readingskills? Evidence from a randomized experiment in 463 classrooms. NBER Working PaperAbstract

There are large gaps in reading skills by family income among school-aged children in the United States. Correlational evidence suggests that reading skills are strongly related to the amount of reading students do outside of school. Experimental evidence testing whether this relationship is causal is lacking. We report the results from a randomized evaluation of a summer reading program called Project READS, which induces students to read more during the summer by mailing ten books to them, one per week. Simple intent-to-treat estimates show that the program increased reading during the summer, and show significant effects on reading comprehension test scores in the fall for third grade girls but not for third grade boys or second graders of either gender. Analyses that take advantage of within-classroom random assignment and cross-classroom variation in treatment effects show evidence that reading more books generates increases in reading comprehension skills, particularly when students read carefully enough to be able to answer basic questions about the books they read, and particularly for girls.

Quinn, D. M., Lynch, K., & Kim, J. S. (Working Paper). Summer Math Interventions and Income Status: A Meta-Analysis.Abstract

Academic summer programs are a popular strategy for providing compensatory education for low income students.  While early meta-analytic evidence suggested that summer school was more effective for middle income students than low income students (Cooper et al., 2000), a recent meta-analysis on summer reading programs found the opposite (Kim & Quinn, 2013).  In this paper, we extend Kim & Quinn (2013) by meta-analyzing 13 studies of summer math programs conducted between 1998 and 2011.  We find, consistent with Kim & Quinn's (2013) reading finding, that summer math programs were more effective for low income students than higher income students.  

2019
Downey, D. B., Quinn, D. M., & Alcaraz, M. (2019). The Distribution of School Quality: Do Schools Serving Mostly White and High-SES Students Produce the Most Learning? Sociology of Education , 92 (4), 386-403. Publisher's VersionAbstract
What is schools’ role in the stratification system? One view is that schools are an important mechanism for perpetuating inequality because children from advantaged backgrounds (white and high socioeconomic) enjoy better school learning environments than their disadvantaged peers. But it is difficult to know this with confidence because children’s development is a product of both school and nonschool factors, making it a challenge to isolate school’s role. A novel approach for isolating school effects is to estimate the difference in learning when school is in versus out, what is called impact. Scholars employing this strategy have come to a remarkable conclusion—that schools serving disadvantaged children produce as much learning as those serving advantaged children. The empirical basis for this position is modest, however, and so we address several shortcomings of the previous research by analyzing a nationally representative sample of about 3,500 children in 270 schools from the Early Childhood Longitudinal Study–Kindergarten Cohort of 2011. With more comprehensive data and better scales, we also find no difference in impact on reading scores across schools serving poor or black children versus those serving nonpoor or white children. These patterns challenge the view that differences in school quality play an important role shaping achievement gaps and prompt us to reconsider theoretical positions regarding schools and inequality.
Quinn, D. M., & Stewart, A. M. (2019). Examining the Racial Attitudes of White Prek-12 Educators. The Elementary School Journal , 120 (2), 272-299. Publisher's VersionAbstract
As the racial/ethnic makeup of the U.S. student population becomes increasingly diverse, the teaching force has remained primarily white. Given this mismatch, and the consequences that educators’ racial attitudes have for students’ academic and developmental outcomes, it is important to understand the racial attitudes of white educators and how those attitudes differ from those of racially minoritized Americans. Using nationally representative data from 1975–2016, we find that compared with racially minoritized respondents, white educators are less supportive of government equalizing efforts, hold more negative racial stereotypes of African Americans, and express more social distance from minoritized groups. Through future research, we must develop a greater understanding of how these attitudes manifest in the classroom to affect students and how problematic attitudes of educators can be improved.
quinn-stewart-racial-attitudes-white-educators-esj-2019.pdf
Quinn, D. M., Desruisseaux, T., & Nkansah-Amankra, A. (2019). "Achievement Gap" Language Affects Teachers' Issue Prioritization. Educational Researcher , 48 (7), 484-487. Publisher's VersionAbstract
The term “achievement gap” is regularly used to describe between-group differences in educational outcomes. However, critics of the term argue that it implies the problem is merely one of student performance and may depress support for policies aimed at structural solutions. We hypothesized that the phrase “racial achievement gap” would elicit lower levels of issue prioritization than the phrase “racial inequality in educational outcomes” due to the latter’s connotations of social justice. In a randomized survey experiment with a national teacher sample (N = 1,549), our hypothesis was confirmed. However, language did not affect teachers’ explanations for existing academic outcome disparities.
quinn-desruisseaux-nkansah-amakra-achievement-gap-language-er-2019.pdf
Kane, T. J., Blazar, D., Gehlbach, H., Greenberg, M., Quinn, D. M., & Thal, D. (2019). Can video technology improve teacher evaluations? An experimental study. Education Finance and Policy. Publisher's VersionAbstract
Teacher evaluation reform has been among the most controversial education reforms in recent years. It also is one of the most costly, in terms of the time teachers and principals must spend on classroom observations. We conducted a randomized field trial in four sites to evaluate whether substituting teacher-collected videos for in-person observations could improve the value of teacher observations for teachers, administrators, or students. Relative to teachers in the control group who participated in standard in-person observations, teachers in the video-based treatment group reported that post-observation meetings were more “supportive” and that they were more able to identify a specific practice they changed afterward. Treatment principals were able to shift their observation work to non-instructional times. The program also substantially increased teacher retention. Nevertheless, the intervention did not improve students' academic achievement or self-reported classroom experiences, either in the year of the intervention or for the next cohort of students. Following from the literature on observation and feedback cycles in low-stakes settings, we hypothesize that to improve student outcomes schools may need to pair video feedback with more specific supports for desired changes in practice.
2018
Quinn, D. M., & Le, Q. T. (2018). Are We Trending to More or Less Between-group Achievement Inequality Over the School Year and Summer? Comparing Across ECLS-K Cohorts. AERA Open , 4 (4), 1-19. Publisher's VersionAbstract
Scholars have argued that schools are “equalizers” because inequalities in test scores by race/ethnicity and socioeconomic status (SES) grow faster over summer vacation than over the school year. In this study, we use nationally representative data from the Early Childhood Longitudinal Study–Kindergarten Classes of 1998–1999 and 2010–2011 to examine the extent to which such patterns have changed over time. Results suggest that more between-group equalizing by race/ethnicity and SES occurred over kindergarten in the recent cohort. However, this was often followed by more inequality widening over summer and more widening or less narrowing over first grade in the latter cohort. The net result was that in recent years, inequality tended to widen more (Black-White) or narrow less (SES and Hispanic-White) over the first 2 years of schooling.
quinn-le-comparing-across-ecls-k-cohorts-aerao-2018.pdf
Quinn, D. M., Kane, T. J., Greenberg, M., & Thal, D. (2018). Effects of a Video-Based Teacher Observation Program on the De-privatization of Instruction: Evidence from a Randomized Experiment. Educational Administration Quarterly. Publisher's VersionAbstract
US schools have traditionally been characterized by teacher privacy and independence, yet theory and empirical work suggest that peer observation and support – or “de-privatized instruction” – can help improve pedagogical practice. In this randomized controlled trial, we investigate whether the introduction of video technology into a school, through a video-based teacher evaluation system called Best Foot Forward (BFF), led to instructional de-privatization. We find that BFF caused administrators to broker more peer support among teachers, made teachers more likely to share lesson videos with colleagues, led teachers to have more of their lessons seen by other teachers, and redistributed which teachers were providing instructional support to colleagues (with relatively newer teachers taking on a larger role in providing peer support). Results suggest that video technology may be an effective tool for efforts to improve instruction by increasing peer observation and support.
quinn-et-al-de-privatization-of-instruction-eaq.pdf
Quinn, D. M., & Kim, J. S. (2018). Experimental Effects of Program Management Approach on Teachers' Professional Ties and Social Capital. Educational Evaluation and Policy Analysis. , 40 (2), 196-218. Publisher's VersionAbstract

Theory and empirical work suggest that teachers’ social capital influences school improvement efforts.  Social ties are prerequisite for social capital, yet little causal evidence exists on how malleable factors, such as instructional management approaches, affect teachers’ ties.  In this cluster-randomized trial, we apply a decision-making perspective to compare a literacy intervention managed under a “fidelity-focused” approach, in which teachers were expected to implement researcher-designed procedures faithfully, versus a “structured adaptive” approach, in which teachers collaboratively planned program adaptations.  In the short term, the adaptive approach increased teachers’ accessing of intervention-related social capital, but decreased their accessing of social capital unrelated to the intervention.  Short-term effects varied based on participants’ role in the intervention.  No group differences were found on social capital measures one year later, suggesting that the structured adaptive approach did not make teachers more likely to form ties that would be useful outside of the intervention. 

quinn-kim-effects-of-program-management-on-teacher-ties-eepa.pdf
2017
Quinn, D. M. (2017). Racial Attitudes of PreK-12 and Postsecondary Educators: Descriptive Evidence from Nationally-representative Data. Educational Researcher , 46, 397-411. Publisher's VersionAbstract
PreK-12 and postsecondary educators’ racial attitudes have important consequences for students’ learning and development, yet we know little about educators’ racial attitudes overall, how their attitudes might differ from those of non-educators, or how attitudes might be changing over time. I investigate these questions using the nationally-representative General Social Survey. Some educators hold worrisome racial attitudes; yet compared to non-educators, educators are less opposed to governmental equalizing efforts, give more politically liberal explanations for racial inequalities, express less negative racial stereotypes, and report less social distance and collective resentments towards minoritized groups. Many educator/non-educator differences were explained by demographics, particularly education level. Time trends mostly show progress in Americans’ racial attitudes, with generally similar trends for educators and non-educators.
quinn-racial-attitudes-of-educators-er-2017.pdf
Kim, J. S., Burkhauser, M. A., Quinn, D. M., Guryan, J., Kingston, H. C., & Aleman, K. (2017). Effectiveness of Structured Teacher Adaptations to an Evidence-based Summer Literacy Program. Reading Research Quarterly. Publisher's VersionAbstract

We conducted a cluster randomized trial to examine the effectiveness of structured teacher adaptations to the implementation of an evidence-based summer literacy program that provided students with (a) books matched to their reading level and interests and (b) teacher scaffolding for summer reading in the form of end-of-year comprehension lessons and materials sent to students’ homes in the summer months. In this study, 27 high-poverty elementary schools (75-100% eligibility for free- or-reduced price lunch) were matched by prior reading achievement and poverty level and randomly assigned to one of two implementation conditions: a core treatment condition that directly replicated implementation procedures used in previous experiments or a core treatment with structured teacher adaptations condition. In the adaptations condition, teachers were organized into grade-level teams around a practical improvement goal and given structured opportunities to use their knowledge, experience, and local data to extend or modify program components for their students and local contexts. Students in the adaptations condition performed .12 standard deviations higher on a reading comprehension posttest than students in the core treatment. An implementation analysis suggests that fidelity to core program components was high in both conditions, while teachers in the adaptations condition primarily made changes that extended or modified program procedures and activities in acceptable ways. Adaptations primarily served to increase the level of family engagement and student engagement with summer books. These results suggest that structured teacher adaptations may enhance rather than diminish the effectiveness of an evidence-based summer literacy program.


 
Quinn, D. M., & Kim, J. S. (2017). Scaffolding Fidelity and Adaptation in Educational Program Implementation: Experimental Evidence from a Literacy Intervention. American Educational Research Journal , 54, 1187-1220. Publisher's VersionAbstract

In a common approach for scaling up effective educational practice, schools adopt evidence-based programs to be implemented with fidelity. An alternative approach assumes that programs should be adapted to local contexts. In this randomized trial of a reading intervention, we study a scaffolded sequence of implementation, in which schools first develop proficiency by implementing the program with fidelity before implementing structured adaptations. We find evidence supporting the scaffolded sequence: a fidelity-focused approach promoted learning and instructional change more so for teachers inexperienced with the intervention, while a structured adaptive approach was more effective for teachers experienced with the intervention. Students benefited more from the structured adaptive approach, but only when their teacher had prior experience with the fidelity-focused version.

quinn-kim-scaffolding-fidelity-and-adaptation-aerj-2017.pdf
Quinn, D. M., & McIntyre, J. (2017). Do Learning Rates Differ by Race/ethnicity over Kindergarten? Reconciling Results across Gain Score, First-Difference, and Random Effects Models. Economics of Education Review , 59, 81-86. Publisher's VersionAbstract

 

The question of whether students’ school-year learning rates differ by race/ethnicity is important for monitoring educational inequality.  Researchers applying different modeling strategies to the same data (the ECLS-K:99) have reached contrasting conclusions on this question.  We outline the similarities and differences across three common approaches to estimating gains and heterogeneity in gains: 1) a gain score model (with intercept), 2) a first-difference (FD) model (in some cases equivalent to regression-through-the-origin [RTO] and student fixed effects models), and 3) a student random effects (RE) model.  We show via simulation that FD/RTO and RE models produce estimates of learning rates, and group differences in learning rates, with more favorable RMSD compared to the gain score model with intercept.  Using data from the ECLS-K:99, we demonstrate that these precision differences lead to contrasting inferences regarding learning rate heterogeneity, and likely explain the inconsistencies across previous studies.     

 

quinn-mcintyre-do-learning-rates-differ-by-race-over-k-eer.pdf
2016
Quinn, D. M., Cooc, N., McIntyre, J., & Gomez, C. J. (2016). Seasonal Dynamics of Academic Achievement Inequality by Socioeconomic Status and Race/ethnicity: Updating and Extending Past Research with New National Data. Educational Researcher , 45, 443-453. Publisher's VersionAbstract

 

Early studies examining seasonal variation in achievement inequality generally concluded that socioeconomic test score gaps grew more over the summer than the school year, suggesting schools served as “equalizers.” In this study, we analyze seasonal trends in SES and racial/ethnic test score gaps using nationally-representative data from the ECLS-K:2011, which includes more school-year and summer rounds than previous national studies.  We further examine how inequality dynamics are influenced by the operationalization of inequality. Findings are consistent with a story in which schools initially accelerate relatively lower-achieving groups’ learning more so than higher-achieving groups; however, this school-year equalizing is not consistently maintained, and sometimes reverses.  When we operationalize inequality as changes in relative position, the reversal of school-year equalizing is more pronounced. 

 

quinn-cooc-mcintyre-gomez-seasonal-dynamics-of-inequality-er.pdf
Kim, J. S., Guryan, J., White, T. G., Quinn, D. M., Capotosto, L., & Kingston, H. C. (2016). The Delayed Effects of a Low-Cost and Large-Scale Summer Reading Intervention on Elementary School Children’s Reading Comprehension. Journal of Research on Educational Effectiveness , 9 (S1), 1-22.Abstract

To improve the reading comprehension outcomes of children in high poverty schools, policymakers need to identify reading interventions that show promise of effectiveness at scale.  This study evaluated the effectiveness of a low-cost and large-scale summer reading intervention that provided comprehension lessons at the end of the school year and stimulated home-based summer reading routines with narrative and informational books.  We conducted a randomized controlled trial involving 59 elementary schools, 463 classrooms, and 6,383 second and third graders and examined outcomes on the North Carolina End-of-Grade (EOG) reading comprehension test administered nine months after the intervention, in the children’s third- or fourth-grade year.  We found that on this delayed outcome, the treatment had a statistically significant impact on children’s reading comprehension, improving performance by .04 SD (standard deviation) overall and .05 SD in high poverty schools. We also found, in estimates from an instrumental variables analysis, that children’s participation in home-based summer book reading routines improved reading comprehension.  The cost-effectiveness ratio for the intervention compared favorably to existing compensatory education programs that target high poverty schools.

2015
Quinn, D.M. & Cooc, N. (2015). Science Achievement Gaps by Gender and Race/Ethnicity in Elementary and Middle School: Trends and Predictors. Educational Researcher , 44 (6), 336-346. Publisher's VersionAbstract

Research on science achievement disparities by gender and race/ethnicity often neglects the beginning of the pipeline in the early grades.  We address this limitation using nationally-representative data following students from grade 3 to 8.  We find that the Black-White science test score gap (-1.07 sd in grade 3) remains stable over these years, the Hispanic-White gap narrows (-.85 to -.65 sd), and the Asian-White grade 3 gap (-.31 sd) closes by grade 8.  The female-male grade 3 gap (-.23 sd) may narrow slightly by eighth grade.  Accounting for prior math and reading achievement, socioeconomic status, and classroom fixed effects, grade 8 racial/ethnic gaps are not statistically significant.  The grade 8 science gender gap disappears after controlling for prior math achievement.

quinn-cooc-science-gaps-er-2015.pdf
Quinn, D. M. (2015). Black-White Summer Learning Gaps: Interpreting the Variability of Estimates across Representations. Educational Evaluation and Policy Analysis , 37 (1), 50-69. Publisher's VersionAbstract

The estimation of racial test score gap trends plays an important role in monitoring educational equality.  Documenting gap trends is complex, however, and estimates can differ depending on the metric, modeling strategy, and psychometric assumptions.  The sensitivity of summer learning gap estimates to these factors has been under-examined.  Using national data, I find Black-White summer gap trends ranging from a significant relative disadvantage for Black students to a significant relative advantage.  Preferred models show no overall gap change the summer after kindergarten, but Black students may make less summer math growth than White students with similar true spring scores.  In estimating gap trends, researchers must recognize that different statistical models not only carry unique assumptions but answer distinct descriptive questions. 

quinn-black-white-summer-learning-gaps-eepa-2015.pdf
Quinn, D. M. (2015). Kindergarten Black-White Test Score Gaps: Re-examining the Roles of Socioeconomic Status and School Quality with New Data. Sociology of Education , 88 (2), 120-139. Publisher's VersionAbstract

Black-white test score gaps form in early childhood and widen over elementary school.  Sociologists have debated the roles that socioeconomic status (SES) and school quality play in explaining these patterns.  In this study I replicate and extend past research using new nationally-representative data from the Early Childhood Longitudinal Study, Kindergarten Class of 2010-2011.  I find black-white test score gaps at kindergarten entry in 2010 in reading (.32 sd), math (.54 sd), and working memory (.52 sd among children with valid scores).  While math and reading gaps widened by approximately .06 sd over kindergarten, the working memory gap was constant.  Multivariate regressions show that student SES explained the reading gap at school entry, but gap decompositions suggest that school quality differences were responsible for reading gap-widening over kindergarten.  SES explained much of the math gap at school entry but math gap-widening could not be explained by SES, school quality, or other hypotheses.  

2013
Kim, J. S., & Quinn, D. M. (2013). The Effects of Summer Reading on Low-Income Children's Literacy Achievement from Kindergarten to Grade 8: A Meta-Analysis of Classroom and Home Interventions. Review of Educational Research , 83, 386-431.Abstract

This meta-analysis reviewed research on summer reading interventions conducted in the United States and Canada from 1998 to 2011. The synthesis 
included 41 classroom- and home-based summer reading interventions involving children from kindergarten to Grade 8. Compared to control group 
children, children who participated in classroom interventions, involving teacher-directed literacy lessons, or home interventions, involving child-initiated 
book reading activities, enjoyed significant improvement on multiple reading outcomes. The magnitude of the treatment effect was positive for summer reading interventions that employed research-based reading instruction and included a majority of low-income children. Sensitivity analyses based on within-study comparisons indicated that summer reading interventions had significantly larger benefits for children from low-income backgrounds than for children from a mix of income backgrounds. The findings highlight the potentially positive impact of classroom- and home-based summer reading interventions on the reading comprehension ability of low-income children.

kim-quinn-summer_reading_meta-analysis-rer-2013.pdf