We present results from a meta-analysis of 37 contemporary experimental and quasi-experimental studies of summer programs in mathematics for children in grades pre-K–12, examining what resources and characteristics predict stronger student achievement. Children who participated in summer programs that included mathematics activities experienced significantly better mathematics achievement outcomes compared to their control group counterparts. We find an average weighted impact estimate of +0.10 standard deviations on mathematics achievement outcomes. We find similar effects for programs conducted in higher- and lower-poverty settings. We undertook a secondary analysis exploring the effect of summer programs on noncognitive outcomes and found positive mean impacts. The results indicate that summer programs are a promising tool to strengthen children’s mathematical proficiency outside of school time.
Many preschool agencies nationwide continue to experience closures and/or conversions to virtual or hybrid instruction due to the ongoing coronavirus disease 2019 (COVID-19) pandemic. Despite the importance of understanding young children’s learning and development during the COVID emergency, limited knowledge exists on adaptable practices for assessing young children during the pandemic. We assess the learning of 336 Head Start children across four states and three different time periods during the 2020 to 2021 school year, using adaptations of traditionally in-person assessments of early numeracy, early literacy, and executive functioning. In doing so, we distill early lessons for the field from the application of a novel, virtual assessment method with the early childhood population. This article describes the adaptations of assessment administration for virtual implementation and incorporation of feedback into continued virtual delivery of assessments. Applications and limitations in broader contexts are discussed.
The COVID-19 pandemic’s impact on preschool children’s school readiness skills remains understudied. This research investigates whether exposure to in-person (versus virtual) Head Start preschool predicted children’s early numeracy, literacy, and executive function outcomes during a pandemic-affected school year, using a novel virtual assessment methodology. Study children (N = 336; mean age = 51 months; 46% Hispanic; 36% Black Non-Hispanic; 52% female) experienced low in-person preschool exposure compared to national pre-pandemic norms. However, study children experienced gains during the pandemic-affected year of 0.08 SD in executive function, 0.34 SD in print knowledge, and 0.49-0.75 in early numeracy skills. For two of the three early numeracy domains measured, spring test score outcomes were stronger among children who attended more in-person preschool.
Despite growing evidence that classroom interventions in science, technology, engineering, and mathematics (STEM) can increase student achievement, there is little evidence regarding how these interventions affect teachers themselves and whether these changes predict student learning. We present results from a meta-analysis of 37 experimental studies of preK-12 STEM professional learning and curricular interventions, seeking to understand how STEM classroom interventions affect teacher knowledge and classroom instruction, and how these impacts relate to intervention impacts on student achievement. Compared with control group teachers, teachers who participated in STEM classroom interventions experienced improvements in content and pedagogical content knowledge and classroom instruction, with a pooled average impact estimate of +0.56 standard deviations. Programs with larger impacts on teacher practice yielded larger effects on student achievement, on average. Findings highlight the positive effects of STEM instructional interventions on teachers, and shed light on potential teacher-level mechanisms via which these programs influence student learning.
How should teachers spend their STEM-focused professional learning time? To answer this question, Heather Hill, Kathleen Lynch, Kathryn Gonzalez, and Cynthia Pollard analyzed a recent wave of rigorous new studies of STEM instructional improvement programs. They found that programs work best when focused on building knowledge teachers can use during instruction. This includes knowledge of the curriculum materials they will use, knowledge of content, and knowledge of how students learn that content. They argue that such learning opportunities improve teachers’ professional knowledge and skill, potentially by supporting teachers in making more informed in-the-moment instructional decisions.
More than half of U.S. children fail to meet proficiency standards in mathematics and science in fourth grade. Teacher professional development and curriculum improvement are two of the primary levers that school leaders and policymakers use to improve children’s science, technology, engineering and mathematics (STEM) learning, yet until recently, the evidence base for understanding their effectiveness was relatively thin. In recent years, a wealth of rigorous new studies using experimental designs have investigated whether and how STEM instructional improvement programs work. This article highlights contemporary research on how to improve classroom instruction and subsequent student learning in STEM. Instructional improvement programs that feature curriculum integration, teacher collaboration, content knowledge, pedagogical content knowledge, and how students learn all link to stronger student achievement outcomes. We discuss implications for policy and practice.
We present results from a meta-analysis of 95 experimental and quasi-experimental pre-K–12 science, technology, engineering, and mathematics (STEM) professional development and curriculum programs, seeking to understand what content, activities, and formats relate to stronger student outcomes. Across rigorously conducted studies, we found an average weighted impact estimate of +0.21 standard deviations. Programs saw stronger outcomes when they helped teachers learn to use curriculum materials; focused on improving teachers’ content knowledge, pedagogical content knowledge, and/or understanding of how students learn; incorporated summer workshops; and included teacher meetings to troubleshoot and discuss classroom implementation. We discuss implications for policy and practice.
For nearly three decades, policy-makers and researchers in the United States have promoted more intellectually rigorous standards for mathematics teaching and learning. Yet, to date, we have limited descriptive evidence on the extent to which reform-oriented instruction has been enacted at scale.
The purpose of the study is to examine the prevalence of reform-aligned mathematics instructional practices in five U.S. school districts. We also seek to describe the range of instruction students experience by presenting case studies of teachers at high, medium and low levels of reform alignment.
We draw on 1,735 video-recorded lessons from 329 elementary teachers in these five U.S. urban districts.
We present descriptive analyses of lesson scores on a mathematics-focused classroom observation instrument. We also draw upon interviews with district personnel, rater-written lesson summaries, and lesson video in order to develop case studies of instructional practice.
We find that teachers in our sample do use reform-aligned instructional practices, but that they do so within the confines of traditional lesson formats. We also find that the implementation of these instructional practices varies in quality. Furthermore, the prevalence and strength of these practices corresponds to the coherence of district efforts at instructional reform.
Conclusions: Our findings suggest that unlike other studies in which reform-oriented instruction rarely occurred (e.g. Kane & Staiger, 2012), reform practices do appear to some degree in study classrooms. In addition, our analyses suggest that implementation of these reform practices corresponds to the strength and coherence of district efforts to change instruction.
Prior research suggests that summer learning loss among low-income children contributes to income-based gaps in achievement and educational attainment. We present results from a randomized experiment of a summer mathematics program conducted in a large, high-poverty urban public school district. Children in the third to ninth grade (N = 263) were randomly assigned to an offer of an online summer mathematics program, the same program plus a free laptop computer, or the control group. Being randomly assigned to the program plus laptop condition caused children to experience significantly higher reported levels of summer home mathematics engagement relative to their peers in the control group. Treatment and control children performed similarly on distal measures of academic achievement. We discuss implications for future research.
Much debate surrounding teacher quality has focused on students’ standardized test scores, but recent federal and state initiatives have emphasized the use of multiple measures to evaluate teacher quality, including classroom observations. In this study, we explore differences across school districts in the relationship between student achievement outcomes and the observed quality of teachers’ instruction. Using data from 298 elementary mathematics teachers in five urban US districts, we examine relationships between teachers’ performance on the Mathematical Quality of Instruction observation instrument and their students’ scores on both state standardized and researcher-developed tests. We find that these relationships differ across school districts. We explore the extent to which differences in skills and expectations for students across tests may explain this variability. An improved understanding of the relationship between classroom observations and student tests may help districts to better support teachers in developing their instructional effectiveness.
Policymakers and researchers have for many years advocated disparate approaches to ensuring teachers deliver high-quality instruction, including requiring that teachers complete specific training requirements, possess a minimum level of content knowledge, and use curriculum materials and professional development resources available from schools and dis-tricts. In this paper, we investigate the extent to which these factors, which we conceptualize as resources for teaching, predict instructional quality in upper elementary mathematics classrooms. Results show that teachers’ mathematical knowledge and their district context explained a moderate share of the variation in mathematics-specific teaching dimensions; other factors, such as teacher experience, preparation, non-instructional work hours, and measures of the school environment, explained very little variation in any dimension.
Mastery of algebra is an important yet difficult milestone for students, suggesting the need for more effective teaching strategies in the algebra classroom. Learning by comparing worked-out examples of algebra problems may be one such strategy. Comparison is a powerful learning tool from cognitive science that has shown promising results in prior small-scale studies in mathematics classrooms. This study reports on a yearlong randomized controlled trial testing the effect of an Algebra I supplemental comparison curriculum on students’ mathematical knowledge. 141 Algebra I teachers were randomly assigned to either implement the comparison curriculum as a supplement to their regular curriculum or to be a ‘business as usual’ control. Use of the supplemental curriculum was much less frequent than requested for many teachers, and there was no main effect of condition on student achievement. However, greater use of the supplemental curriculum was associated with greater procedural student knowledge. These findings suggest a role for comparison in the algebra classroom but also the challenges of supporting teacher integration of new materials into the curriculum.
Discussions where teachers engage students in the comparison of multiple solution strategies to a single problem have been recommended in curriculum policy documents, yet integrating these discussions into teachers’ normative routines is not widespread. In this paper, we begin to explore variations in teachers’ implementation of Algebra I curriculum materials specifically focused on comparison. We explore (via case studies) implementation of the curriculum materials by two teachers with similar teaching backgrounds. The case studies suggest that these two teachers’ implementation of the comparison materials differed markedly, raising questions about possible factors which may have contributed to implementation differences.
Despite extensive scholarship about the importance of teaching mathematics with multiple strategies in the elementary grades, there has been relatively little discussion of this practice in the middle and high school levels or in the context of introductory algebra. This article begins our exploration of this practice by addressing the following questions: (1) What do middle and high school Algebra I teachers describe as the advantages of instruction that includes a focus on multiple strategies?; and (2) What disadvantages to this practice do these teachers describe? Our analysis, based on the data from interviews (N= 13) and surveys (N= 79) conducted with experienced middle and secondary mathematics teachers, indicates that middle and secondary math teachers’ reported views surrounding multiple strategies appear to differ in important ways from those typically associated with teaching with multiple strategies in the elementary grades.
Although policy documents promote teaching students multiple strategies for solving mathematics problems, some practitioners and researchers argue that struggling learners will be confused and overwhelmed by this instructional practice. In the current exploratory study, the authors explore how 6 struggling students viewed the practice of learning multiple strategies at the end of a yearlong algebra course that emphasized this practice.
Measurement scholars have recently constructed validity arguments in support of a variety ofeducational assessments, including classroom observation instruments. In this article, we note thatusers must examine the robustness of validity arguments to variation in the implementation ofthese instruments. We illustrate how such an analysis mightbe used to assess a validity argumentconstructed for the Mathematical Quality of Instruction instrument, focusing in particular on the20effects of varying the rater pool, subject matter content, observation procedure, and district context.Variation in the subject matter content of lessons did not affect rater agreement with master scores,but the evaluation of other portions of the validity argument varied according to the compositionof the rater pool, observation procedure, and district context. These results demonstrate the needfor conducting such analyses, especially for classroom observation instruments that are subject to25multiple sources of variation.
The ability to flexibly solve problems is considered an important outcome for school mathematics and is the focus of this paper. The paper describes the impact of a three-week summer course for students who struggle with algebra. During the course, students regularly compared and contrasted worked examples of algebra problems in order to promote flexible use of solution strategies. Assessments were designed to capture both knowledge and use of multiple strategies. The students were interviewed in order to understand their rationales for choosing particular strategies as well as their attitudes toward instruction that emphasized multiple strategies. Findings suggest that students gained both knowledge of and appreciation for multiple strategies, but they did not always use alternate strategies. Familiarity, understandability, efficiency, and form of the problem were all considerations for strategy choice. Practical and theoretical implications are discussed.
The ability to estimate is a fundamental real-world skill; it allows students to check the reasonableness of answers found through other means, and it can help students develop a better understanding of place value, mathematical operations, and general number sense. Flexibility in the use of strategies is particularly critical in computational estimation. The ability to perform complex calculations mentally is cognitively challenging for many students; thus, it is important to have a broad repertoire of estimation strategies and to select the most appropriate strategy for a given problem. In this paper, we consider the role of students’ prior knowledge of estimation strategies in the effectiveness of interventions designed to promote strategy flexibility across two recent studies. In the first, 65 fifth graders began the study as fluent users of one strategy for computing mental estimates to multi-digit multiplication problems such as 17 × 41. In the second, 157 fifth and sixth graders began the study with moderate to low prior knowledge of strategies for computing mental estimates. Results indicated that students’ fluency with estimation strategies had an impact on which strategies they adopted. Students who exhibited high fluency at pretest were more likely to increase use of estimation strategies that led to more accurate estimates, while students with less fluency adopted strategies that were easiest to implement. Our results suggest that both the ease and accuracy of strategies as well as students’ fluency with strategies are all important factors in the development of strategy flexibility.