Working Paper
Kraft MK, Hill HC. Developing Ambitious Mathematics Instruction through Web-Based Coaching: A Randomized Field Trial. Working Paper.Abstract

This paper describes and evaluates a web-based coaching program designed to support teachers in implementing Common Core-aligned math instruction.  Web-based coaching programs can be operated at relatively lower costs, are scalable, and make it more feasible to pair teachers with coaches who have expertise in their content area and grade level.  Results from our randomized field trial document sizable and sustained effects on teachers’ ability to analyze instruction and on their instructional practice, with the latter measured by scores on the Mathematical Quality of Instruction (MQI) rubric and student surveys.  These impacts did not result in increases in student achievement as measured by state standardized tests or supplemental formative assessments in math, although we cannot rule out the possibility of small effects.

Download pdf here
Kraft MA, Papay JP, Chi OL. Teacher skill development: Evidence from performance ratings by principals. Working Paper.Abstract

We examine the dynamic nature of teacher skill development using panel data on principals’ subjective performance ratings of teachers. Past research on teacher productivity improvement has focused primarily on one important but narrow measure of performance: teachers’ value-added to student achievement on standardized tests. Unlike value-added, subjective performance ratings provide detailed information about specific dimensions of teacher skills and are available for the many teachers in non-tested grades and subjects. Using a within-teacher returns to experience framework, we find, on average, large and rapid improvements in teachers’ instructional practices throughout their first ten years on the job as well as substantial differences in improvement rates across individual teachers. We also document that subjective performance ratings contain important information about teacher effectiveness. In the district we study, principals appear to successfully differentiate teacher performance throughout the full distribution instead of just in the tails. Furthermore, prior performance ratings and gains in these ratings provide additional information about teachers’ ability to improve test scores that is not captured by prior value-added scores. Taken together, our study provides new insights on teacher performance improvement and variation in teacher development across instructional skills and individual teachers.

Download pdf here
Kraft MA, Brunner EJ, Dougherty SM, Schwegman DJ. Teacher Evaluation Reforms and the Supply and Quality of New Teachers. Working Paper.Abstract

In recent years, states have sought to increase accountability for public school teachers by implementing high-stakes evaluation systems. We examine the effect of these reforms on the supply and quality of new teachers. Leveraging variation across states and time, we find that evaluation reforms reduced the supply of new teaching candidates by 17 percent and increased the likelihood of unfilled teaching positions, particularly in hard-to-staff schools. Reforms also increased the quality of newly hired teachers by shifting the lower tail of the distribution upward. We find evidence that decreased job security, satisfaction, and autonomy are likely mechanisms for these effects.

Download pdf here
Kraft MA, Brunner EJ, Dougherty SM, Schwegman D. Teacher Accountability Reforms and the Supply of New Teachers. Working Paper.Abstract

In recent years, states across the country have attempted to increase the accountability of public school teachers by implementing rigorous, high-stakes evaluation systems and in some cases repealing teacher tenure protections. We examine the effect of these reforms on the supply of new entrants into the teacher labor market by exploiting a unique panel dataset that includes the number of teaching licenses granted by states. Leveraging variation in the adoption of reforms across states and time, we find that evaluation reforms resulted in a steady decline in the statewide supply of new teachers, whereas tenure reforms produced a sharp but more temporary contraction. In exploratory analyses, we find mixed evidence of the effect of accountability on the selectivity of the institutions where prospective teachers earned their teaching degrees. There is little evidence evaluation reforms had any differential effect by university selectivity, while tenure reforms appear to have reduced supply more among candidates from less selective universities. We find no evidence that decreases in labor supply were concentrated in non-shortage or shortage licensure areas.

Download pdf here
Kraft MA. Teacher Effects on Complex Cognitive Skills and Social-Emotional Competencies. Journal of Human Resources. Forthcoming.Abstract

I exploit the random assignment of class rosters in the MET Project to estimate teacher effects on students’ performance on complex open-ended tasks in math and reading, as well as their growth mindset, grit, and effort in class. I find large teacher effects across this expanded set of outcomes, but weak relationships between these effects and performance measures used in current teacher evaluation systems including value-added to state standardized tests. These findings suggest teacher effectiveness is multidimensional and high-stakes evaluation decisions are only weakly informed by the degree to which teachers are developing students’ complex cognitive skills and social-emotional competencies.

Download pdf here
Kraft MA, Blazar D. Taking Teacher Coaching To Scale: Can Personalized Training Become Standard Practice? . Education Next [Internet]. 2018;18 (4). Publisher's Version
Kraft MA, Blazar D, Hogan D. The Effect of Teacher Coaching on Instruction and Achievement: A Meta-Analysis of the Causal Evidence. Review of Educational Research [Internet]. 2018;88 (4) :547-588. Publisher's VersionAbstract

Teacher coaching has emerged as a promising alternative to traditional models of professional development.  We review the empirical literature on teacher coaching and conduct meta-analyses to estimate the mean effect of coaching programs on teachers’ instructional practice and students’ academic achievement.  Combining results across 60 studies that employ causal research designs, we find pooled effect sizes of 0.49 standard deviations (SD) on instruction and 0.18 SD on achievement.  Much of this evidence comes from literacy coaching programs for pre-kindergarten and elementary school teachers.  Although these findings affirm the potential of coaching as a development tool, further analyses illustrate the challenges of taking coaching programs to scale while maintaining effectiveness.  Average effects from effectiveness trials of larger programs are only a fraction of the effects found in efficacy trials of smaller programs. We conclude by discussing ways to address scale-up implementation challenges and providing guidance for future causal studies.

Download pdf here
Papay JP, Kraft MK. Developing Workplaces Where Teachers Stay, Improve, and Succeed. In: Teaching in Context: How Social Aspects of School and School Systems Shape Teachers’ Development & Effectiveness. Cambridge: Harvard Education Press ; 2017. pp. 15-35. Publisher's Version Download pdf here
Kraft MA. Engaging Parents Through Better Communication Systems. Educational Leadership [Internet]. 2017;71 (1) :58-62. Publisher's VersionAbstract

New research shows that frequent, personalized outreach to parents works. How can schools support the practice?

Download pdf here
Kraft MA, Monti-Nussbaum M. Can schools enable parents to prevent summer learning loss? A text messaging field experiment to promote literacy skills. The ANNALS of the American Academy of Political and Social Science [Internet]. 2017;674 (1) :85-112. Publisher's VersionAbstract

The vast differences in summer learning activities among children present a substantial challenge to providing equal educational opportunity in the United States. Most initiatives
aimed at reversing summer learning loss focus on school- or center-based programs. This study explores the potential of enabling parents to provide literacy development opportunities at home as a low-cost alternative. We conduct a randomized field trial of a summer text-messaging pilot program for parents focused on promoting literacy skills among first through fourth graders. We find positive effects on reading comprehension among third and fourth graders, with effect sizes of .21 to .29 standard deviations, but no effects for first and second graders. Texts also increased attendance at parent-teacher conferences but not at other school-related activities. Evidence to inform future efforts to reverse summer learning loss is
provided by parents’ responses to a follow-up survey.

Download pdf here
Steinberg M, Kraft MA. The Sensitivity of Teacher Performance Ratings to the Design of Teacher Evaluation Systems. Educational Researcher [Internet]. 2017;46 (7) :378-396. Publisher's VersionAbstract

In recent years, states and districts have responded to federal incentives by instituting major reforms to their teacher evaluation systems. The passage of the Every Student Succeeds Act in 2015 now provides policymakers with even greater autonomy to redesign existing evaluation systems. Yet, little evidence exists to inform decisions about two key system design features – teacher performance measure weights and performance ratings thresholds. Using data from the Measures of Effective Teaching study, we conduct simulation-based analyses that illustrate the critical role that performance measure weights and ratings thresholds play in determining teachers’ summative evaluation ratings and the distribution of teacher proficiency rates. These findings offer insights to policymakers and administrators as they refine and possibly remake teacher evaluation systems. 

Download pdf here
Blazar D, Kraft MA. Teacher and teaching effects on students’ attitudes and behaviors. Educational Evaluation and Policy Analysis [Internet]. 2017;39 (1) :146-170. Publisher's VersionAbstract


Research has focused predominantly on how teachers affect students’ achievement on tests despite evidence that a broad range of attitudes and behaviors are equally important to their long-term success. We find that upper-elementary teachers have large effects on self-reported measures of students’ self-efficacy in math, and happiness and behavior in class. Students’ attitudes and behaviors are predicted by teaching practices most proximal to these measures, including teachers’ emotional support and classroom organization. However, teachers who are effective at improving test scores often are not equally effective at improving students’ attitudes and behaviors. These findings lend empirical evidence to well-established theory on the multidimensional nature of teaching and the need to identify strategies for improving the full range of teachers’ skills.

Download pdf here
Charner-Laird M, Ng M, Johnson SM, Kraft MA, Papay JP, S.K R. Gauging goodness of fit: Teachers’ expectations for their instructional teams in high-poverty schools. American Journal of Education [Internet]. 2017;123 (4) :553-584. Publisher's VersionAbstract

Teacher teams are increasingly common in urban schools. Here we analyze teachers'
responses to teams in six high-poverty schools. Teachers used two criteria to assess
teams' "goodness of fit" in meeting the demands of their work—whether their team
helped them teach better and whether it contributed to a better school. Their responses
differed notably by school, depending largely on the principal's approach to
implementation. In the three schools where teachers assessed teams favorably,
principals set a meaningful purpose for teachers' collaborative work, contributed
structural and professional expertise for their deliberations, and established a safe
environment for teachers' on-the-job growth.

Kraft MA, Gilmour AF. Revisiting the Widget Effect: Teacher Evaluation Reforms and the Distribution of Teacher Effectiveness. Educational Researcher [Internet]. 2017;46 (5) :234-249. Publisher's VersionAbstract

In 2009, The New Teacher Project (TNTP)’s The Widget Effect documented the failure to recognize and act on differences in teacher effectiveness. We revisit these findings by compiling teacher performance ratings across 24 states that adopted major reforms to their teacher evaluation systems. In the vast majority of these states, the percentage of teachers rated Unsatisfactory remains less than 1%. However, the full distributions of ratings vary widely across states with 0.7% to 28.7% rated below Proficient and 6% to 62% rated above Proficient. We present original survey data from an urban district illustrating that evaluators perceive more than three times as many teachers in their schools to be below Proficient than they rate as such. Interviews with principals reveal several potential explanations for these patterns.

Download pdf here
Kraft MA, Blazar DL. Individualized Coaching to Improve Teacher Practice Across Grades and Subjects: New Experimental Evidence. Educational Policy [Internet]. 2017;31 (7) :1033-1068. Publisher's VersionAbstract

This paper analyzes a coaching model focused on classroom management skills and instructional practices across grade levels and subject areas. We describe the design and implementation of MATCH Teacher Coaching among an initial cohort of fifty-nine teachers working in New Orleans charter schools.  We evaluate the effect of the program on teachers’ instructional practices using a block randomized trial and find that coached teachers scored 0.59 standard deviations higher on an index of effective teaching practices comprised of observation scores, principal evaluations, and student surveys. We discuss implementation challenges and make recommendations for researcher-practitioner partnerships to address key remaining questions.

Download pdf here
Papay JP, Kraft MA. The Myth of the Teacher Performance Plateau. Educational Leadership [Internet]. 2016;73 (8) :36-42. Publisher's VersionAbstract

It’s almost accepted as fact that teachers don’t improve after their first few years on the job. New research challenges this common assumption.

Download pdf here
West MR, Kraft MA, Finn AS, Martin RE, Duckworth AL, Gabrieli CFO, Gabrieli JDE. Promise and Paradox: Measuring Students’ Non-cognitive Skills and the Impact of Schooling. Educational Evaluational and Policy Analysis [Internet]. 2016;38 (1) :148-170. Publisher's VersionAbstract

We used self-report surveys to gather information on a broad set of non-cognitive skills from 1,368 8th-graders. At the student level, scales measuring conscientiousness, self-control, grit, and growth mindset are positively correlated with attendance, behavior, and test-score gains between 4th- and 8th-grade. Conscientiousness, self-control, and grit are unrelated to test-score gains at the school level, however, and students attending over-subscribed charter schools score lower on these scales than do students attending district schools. Exploiting admissions lotteries, we find positive impacts of charter school attendance on achievement and attendance but negative impacts on these non-cognitive skills. We provide suggestive evidence that these paradoxical results are driven by reference bias, or the tendency for survey responses to be influenced by social context.

Download pdf here
Papay JP, Kraft MA. The productivity costs of inefficient hiring practices: Evidence from late teacher hiring. Journal of Policy Analysis and Management [Internet]. 2016;35 (4) :791-817. Publisher's VersionAbstract

We use matched employee-employer records from the teacher labor market to explore the trade-offs between the timing of hiring and match quality. Hiring teachers after the school year starts reduces student achievement by 0.042SD in mathematics and 0.026SD in reading. This reflects, in part, a temporary disruption effect in the first year. In mathematics, but not in reading, late-hired teachers remain persistently less effective, evidence of negative selection in the teacher labor market. Late hiring concentrates in schools that disproportionately serve disadvantaged student populations, contributing to challenges in ensuring an equitable distribution of educational resources across students.

Download pdf here
Kraft MA, Gilmour A. Can Principals Promote Teacher Development as Evaluators? A Case Study of Principals’ Views and Experiences. Educational Administration Quarterly [Internet]. 2016;52 (5) :711-753. Publisher's VersionAbstract

Purpose: New teacher evaluation systems have expanded the role of principals as instructional leaders, but little is known about principals’ ability to promote teacher development through the evaluation process. We conducted a case study of principals’ perspectives on evaluation and their experiences implementing observation and feedback cycles to better understand whether principals feel as though they are able to promote teacher development as evaluators.


Research Methods: We conducted interviews with a stratified random sample of 24 principals in an urban district that recently implemented major reforms to its teacher evaluation system. We analyzed these interviews by drafting thematic summaries, coding interview transcripts, creating data-analytic matrices, and writing analytic memos.


Findings: We found that the evaluation reforms provided a common framework and language that helped facilitate principals’ feedback conversations with teachers. However, we also found that tasking principals with primary responsibility for conducting evaluations resulted in a variety of unintended consequences which undercut the quality of evaluation feedback they provided. We analyze five broad solutions to these challenges: strategically targeting evaluations, reducing operational responsibilities, providing principal training, hiring instructional coaches, and developing peer evaluation systems.


Implications: The quality of feedback teachers receive through the evaluation process depends critically on the time and training evaluators have to provide individualized and actionable feedback. Districts that task principals with primary responsibility for conducting observation and feedback cycles must attend to the many implementation challenges associated with this approach in order for next-generation evaluation systems to successfully promote teacher development.

Download pdf here
Kraft MA, Marinell WM, Yee D. School organizational contexts, teacher turnover, and student achievement: Evidence from panel data. American Educational Research Journal [Internet]. 2016;53 (5) :1411-1499. Publisher's VersionAbstract

We study the relationship between school organizational contexts, teacher turnover, and student achievement in New York City (NYC) middle schools. Using factor analysis, we construct measures of four distinct dimensions of school contexts captured on the annual NYC School Survey. We identify credible estimates by isolating variation in organizational contexts within schools over time. We find that improvements in school leadership, academic expectations, teacher relationships, and school safety are all independently associated with corresponding reductions in teacher turnover. Increases in school safety and academic expectations for students also correspond to increases in student achievement. These results are robust to a range of potential threats to validity suggesting that our findings are likely driven by an underlying causal relationship.

Download pdf here