Context: A perennial problem in applied statistics is the inability to justify strong claims about cause-and-effect relationships without full knowledge of the mechanism determining selection into treatment. Few research designs other than the well-implemented random assignment study meet this requirement. Researchers have proposed partial…

Background: Race to the Top era teacher evaluation reforms sought to improve teacher effectiveness and, ultimately, student achievement through either accountability or developmental mechanisms (Donaldson, 2021). Quasi-experimental teacher evaluation studies find mixed results in urban and large-scale settings (Dee & Wyckoff, 2015; Steinberg…

Background/Context: For decades, practitioners, policymakers, and researchers have sought to improve the quality of early childhood education (ECE) to foster equitable experiences for young children. States are increasingly investing in evidence-based professional development programs for teachers, including practice-based coaching and early…

Background: Over the past 20 years, much has been written about the merits and limitations of matching methods, particularly propensity score matching (see, e.g., Stuart, 2010 for a review of matching approaches), and the myriad ways matching can be implemented to identify a comparison group that is similar to the treatment group on observable…

Background & Context: Early childhood contexts characterized by high-quality adult-child interactions have important consequences for children's development (Hamre, 2014; Peisner-Feinberg et al., 2001). The use of classroom observation tools to measure the quality of such interactions in early education and care (EEC) settings has become…

Observational studies are common in educational research, where subjects self-select or are otherwise non-randomly assigned to different interventions (e.g., educational programs, grade retention, special education). Unbiased estimation of a causal effect with observational data depends crucially on the assumption of ignorability, which specifies…

Over the past decade, states and districts have updated teacher evaluation systems in response to federal incentives, such as waivers to No Child Left Behind and grants from Race to the Top. The use of multiple measures, however, raises an important issue for policymakers and practitioners: to what extent should different measures of a teacher's…

Classroom observation protocols, in which observers rate multiple dimensions of teaching according to established protocols (either live in the classroom, or post-hoc from lesson videos), are increasingly being used in both research and policy contexts. However, scores generated from these protocols have many sources of error. Day to day variation…

Efficacy studies of educational programs often involve mediator analyses aimed at testing empirically appropriate theories of action. In particular, in the studies of professional teacher development programs, the intervention targets presumably teacher performance while the ultimate outcome is the student achievement measured by a standardized…

Policy changes from the past decade have resulted in a growing interest in identifying effective teachers and their characteristics. This study is the third study to use data from a randomized experiment to test the validity of measures of teacher effectiveness. The authors collected effectiveness measures across three school years from three…

Recent research on multiple measures of teaching effectiveness has redefined the role of in-classroom observations in teacher evaluation systems. In particular, most states now mandate that teachers are observed on multiple occasions during the school year, and it is increasingly common that multiple raters are utilized across the different rating…

Rapid development of sustained attention occurs during infancy and toddlerhood, as neurological maturation allows children to increasingly attend to objects and events in the environment (Ruff & Rothbart, 1996). Play experiences during this period can serve as a context during which children's ability to attend can provide an opportunity for…

In contrast to randomized experiments, the estimation of unbiased treatment effects from observational data requires an analysis that conditions on all confounding covariates. Conditioning on covariates can be done via standard parametric regression techniques or nonparametric matching like propensity score (PS) matching. The regression or…

This study evaluates the impact of a group (school-based) teacher bonus program introduced in the Brazilian state of Pernambuco in 2008. Research questions are: i) Does setting school level targets and awarding bonus pay produce improvements in student learning? ii) Does setting school level targets and awarding bonus pay stimulate visible…

As the education reform movement increasingly focuses on teachers and teaching, educators, policy-makers, and researchers need valid and reliable measures that can be used to evaluate individual teachers, provide guidance for improving teaching performance, and support research in ways that advance instruction and classroom dialog and practice. A…