Quinn, D. M., & Ho, A. D. (2020).
Ordinal Approaches to Decomposing Between-group Test Score Disparities.
Journal of Educational and Behavioral Statistics.
Publisher's VersionAbstractThe estimation of test score “gaps” and gap trends plays an important role in monitoring educational inequality. Researchers decompose gaps and gap changes into within- and between-school portions to generate evidence on the role schools play in shaping these inequalities. However, existing decomposition methods assume an equal-interval test scale and are a poor fit to coarsened data such as proficiency categories. This leaves many potential data sources ill-suited for decomposition applications. We develop two decomposition approaches that overcome these limitations: an extension of V, an ordinal gap statistic, and an extension of ordered probit models. Simulations show V decompositions have negligible bias with small within-school samples. Ordered probit decompositions have negligible bias with large within-school samples but more serious bias with small within-school samples. More broadly, our methods enable analysts to (1) decompose the difference between two groups on any ordinal outcome into portions within- and between some third categorical variable and (2) estimate scale-invariant between-group differences that adjust for a categorical covariate.
quinn-ho-ordinal-decomposition-jebs-2020.pdf Copur-Gencturk, Y., Thacker, I., & Quinn, D. M. (2020).
K-8 Teachers’ Overall and Gender-Specific Beliefs About Mathematical Aptitude.
International Journal of Science and Mathematics Education .
Publisher's VersionAbstractTeachers' beliefs play a significant role in students' academic attainment and career choices. Despite comparable attainment levels between genders, persistent stereotypes and beliefs that certain disciplines require innate ability and that men and women have different ability levels impede students' academic career paths. In this study, we examined the prevalence of U.S. mathematics teachers' explicit general and gender-specific beliefs about mathematical ability and identified which teacher characteristics were associated with these beliefs. An analysis of data from 382 K-8 teachers in the USA indicated that overall, teachers disagreed with the idea that general and gender-specific mathematical ability is innate, and agreed with the idea that hard work and dedication are required for success in mathematics. However, our findings indicate that those who believed mathematics requires brilliance also tended to think girls lacked this ability. We also found that teachers who were teaching mathematics to 11-to 14-year old students seemed to believe that mathematics requires innate ability compared with teachers who were teaching mathematics to 5-to 10-year-old students. In addition, more experienced teachers and teachers who worked with special education students seemed to believe less in the role of hard work in success in mathematics, which could have serious consequences for shaping their students' beliefs about their academic self-concept and future career-related decisions.
Chin, M. J., Quinn, D. M., Dhaliwal, T. K., & Lovison, V. S. (2020).
Bias in the Air: A Nationwide Exploration of Teachers' Implicit Racial Attitudes, Aggregate Bias, and Student Outcomes. Educational Researcher.
Publisher's VersionAbstractTheory suggests that teachers’ implicit racial attitudes affect their students, but large-scale evidence on U.S. teachers’ implicit biases and their correlates is lacking. Using nationwide data from Project Implicit, we found that teachers’ implicit White/Black biases (as measured by the implicit association test) vary by teacher gender and race. Teachers’ adjusted bias levels are lower in counties with larger shares of Black students. In the aggregate, counties in which teachers hold higher levels of implicit and explicit racial bias have larger adjusted White/Black test score inequalities and White/Black suspension disparities.
chin-quinn-dhaliwal-lovison-bias-in-the-air-er-2020.pdf Quinn, D. M. (2020).
Experimental Evidence on Teachers' Racial Bias in Student Evaluation: The Role of Grading Scales.
Educational Evaluation and Policy Analysis ,
42, 375-392.
Publisher's VersionAbstractA vast research literature documents racial bias in teachers’ evaluations of students. Theory suggests bias may be larger on grading scales with vague or overly general criteria versus scales with clearly specified criteria, raising the possibility that well-designed grading policies may mitigate bias. This study offers relevant evidence through a randomized Web-based experiment with 1,549 teachers. On a vague grade-level evaluation scale, teachers rated a student writing sample lower when it was randomly signaled to have a Black author, versus a White author. However, there was no evidence of racial bias when teachers used a rubric with more clearly defined evaluation criteria. Contrary to expectation, I found no evidence that the magnitude of grading bias depends on teachers’ implicit or explicit racial attitudes.
quinn-racial-bias-grading-eepa-2020.pdf Quinn, D. M. (2020).
Experimental effects of "achievement gap" news reporting on viewers' racial stereotypes, inequality explanations, and inequality prioritization.
Educational Researcher ,
49, 482-492.
Publisher's VersionAbstractThe “achievement gap” has long dominated mainstream conversations about race and education. Some scholars warn that the discourse around racial gaps perpetuates stereotypes and promotes the adoption of deficit-based explanations that fail to appreciate the role of structural inequities. I investigate through three randomized experiments. Results indicate that a TV news story about racial achievement gaps (vs. a control or counterstereotypical video) led viewers to express more exaggerated stereotypes of Black Americans as lacking education (Study 1 effect size = .30 SD; Study 2 effect size = .38 SD) and may have increased viewers’ implicit stereotyping of Black students as less competent than White students (Study 1 effect size = .22 SD; Study 2 effect size = .12 SD, ns). The video did not affect viewers’ explicit competence-related racial stereotyping, the explanations they gave for achievement inequalities, or their prioritization of ending achievement inequalities. After 2 weeks, the effect on stereotype exaggeration faded. Future research should probe how we can most productively frame educational inequality by race.
quinn-effects-of-ag-news-reporting-2020.pdf Kane, T. J., Blazar, D., Gehlbach, H., Greenberg, M., Quinn, D. M., & Thal, D. (2020).
Can video technology improve teacher evaluations? An experimental study. Education Finance and Policy ,
15, 397-427.
Publisher's VersionAbstractTeacher evaluation reform has been among the most controversial education reforms in recent years. It also is one of the most costly, in terms of the time teachers and principals must spend on classroom observations. We conducted a randomized field trial in four sites to evaluate whether substituting teacher-collected videos for in-person observations could improve the value of teacher observations for teachers, administrators, or students. Relative to teachers in the control group who participated in standard in-person observations, teachers in the video-based treatment group reported that post-observation meetings were more “supportive” and that they were more able to identify a specific practice they changed afterward. Treatment principals were able to shift their observation work to non-instructional times. The program also substantially increased teacher retention. Nevertheless, the intervention did not improve students' academic achievement or self-reported classroom experiences, either in the year of the intervention or for the next cohort of students. Following from the literature on observation and feedback cycles in low-stakes settings, we hypothesize that to improve student outcomes schools may need to pair video feedback with more specific supports for desired changes in practice.