Background: Teacher quality matters for student achievement (Coleman, 1968; Rivkin et al., 2005; Rockoff, 2004; Aaronson et al., 2007) and later career success (Chetty et al., 2014). States use licensure exams as a quality screen believing that they are predictive of teaching effectiveness (Council et al., 2001). However, the evidence on the…

This report provides advice to policymakers and school leaders on the use of assessment centres as part of a teacher selection approach. It discusses the relationship between assessment centre scores prior to joining teaching, and teacher effectiveness over a six-year period. It draws from various stages of a wider research project, the overall…

As students are exposed to extreme temperatures with ever-increasing frequency, it is important to understand how such exposure affects student learning. In this paper we draw upon detailed student achievement data, combined with high-resolution weather records, to paint a clear portrait of the effect of temperature on student learning across a…

We provide a novel characterization of augmented balancing weights, also known as automatic debiased machine learning (AutoDML). These popular "doubly robust" or "de-biased machine learning estimators" combine outcome modeling with balancing weights -- weights that achieve covariate balance directly in lieu of estimating and…

We describe the development of 236 children with Angelman syndrome (AS) using the Bayley Scales of Infant and Toddler Development, Third Edition. Multilevel linear mixed modeling approaches were used to explore differences between molecular subtypes and over time. Individuals with AS continue to make slow gains in development through at least age…

Our study examines the differential item functioning of the Counselor Competencies Scale--Revised (CCS-R) scores due to respondents' gender, the type of evaluation, and a combination of these two variables using a large sample (N = 1614). Implications of the findings are offered to inform counselor educators and supervisors using the CCS-R and…

In order to detect a wide range of aberrant behaviors, it can be useful to incorporate information beyond the dichotomous item scores. In this paper, we extend the l[subscript z] and l*[subscript z] person-fit statistics so that unusual behavior in item scores and unusual behavior in item distractors can be used as indicators of aberrance. Through…

Validity is a fundamental consideration of test development and test evaluation. The purpose of this study is to define and reify three key aspects of validity and validation, namely test-score interpretation, test-score use, and the claims supporting interpretation and use. This study employed a Delphi methodology to explore how experts in…

The use of the aggregate scoring method for scoring concordance tests requires the weighting of test items to be derived from the performance of a group of experts who take the test under the same conditions as the examinees. However, the average score of experts constituting the reference panel remains a critical issue in the use of these tests.…

The part of responses that is absent in the nonequivalent groups with anchor test (NEAT) design can be managed to a planned missing scenario. In the context of small sample sizes, we present a machine learning (ML)-based imputation technique called chaining random forests (CRF) to perform equating tasks within the NEAT design. Specifically, seven…

This paper proposes to evaluate learning efficiency by implementing the flipped classroom and automatic source code evaluation based on the Kirkpatrick evaluation model in students of CS1 programming course. The experimentation was conducted with 82 students from two CS1 courses; an experimental group (EG = 56) and a control group (CG = 26). Each…

Though a substantial amount of research exists on imputing missing scores in educational assessments, there is little research on cases where responses or scores to an item are missing for all test takers. In this paper, we tackled the problem of imputing missing scores for tests for which the responses to an item are missing for all test takers.…

The Annual Professional Performance Review (APPR) system in New York State (NYS) has been called into question by educators since its adoption nearly 10 years ago, yet it remains the mandated evaluation system in NYS schools today. Much of the concern has been over changes such as assigning teachers final evaluation scores, as well as for the…

We investigate the effects of private schools on reading and numeracy scores using rich population data. Conditional on lagged test scores and narrowly defined neighborhood indicators, Catholic and non-Christian faith private schools on average raise test scores by 0.18 standard deviation or more relative to the average public school, while…

Background: The generalizability of a study's results continues to be at the forefront of concerns in evaluation research in education (Tipton & Olsen, 2018). Over the past decade, statisticians have developed methods, mainly based on propensity scores, to improve generalizations in the absence of random sampling (Stuart et al., 2011; Tipton,…