To evaluate how Advanced Placement (AP) courses affect college-going, we randomly assigned the offer of enrollment into an AP science course to over 1,800 students in twenty-three schools that had not previously offered the course. We find no AP course effects on students' college entrance exam scores (SAT/ACT). As expected, AP course-takers are…

Cloze items are commonly used for both assessing learning and as a learning activity. This paper investigates the selection of sentences for cloze item creation by comparing methods ranging from simple heuristics to deep learning summarization models. An evaluation using human-generated cloze items from three different science texts indicates that…

The purpose of this study is to investigate how Harmony Public Schools' (HPS) ' Class of 2024 high school students' current selves (high school experience, math and science efficacy, parent, and teacher expectations) affect to possible future selves (i.e. choosing a science, technology, engineering, and mathematics [STEM]-major in college). 1464…

In this commentary, University of North Carolina's associate dean of research and assessment at the School of Education Terry Ackerman poses questions and shares his thoughts on David Thissen's essay, "Bad Questions: An Essay Involving Item Response Theory" (this issue). Ackerman begins by considering the two purposes of Item Response…

This quasiexperiment tests the effect of the use of a randomized name generator (wheel of discussion) on course performance in an introductory psychology course. In one section of the course, the instructor implemented the use of the wheel of discussion immediately, whereas, in the second section, the instructor introduced the wheel of discussion…

The intent of this research was to find an item selection procedure in the multidimensional computer adaptive testing (CAT) framework that yielded higher precision for both the domain and composite abilities, had a higher usage of the item pool, and controlled the exposure rate. Five multidimensional CAT item selection procedures (minimum angle;…

The specifics of how growth models should be constructed and used to evaluate schools and teachers is a topic of lively policy debate in states and school districts nationwide. In this paper we take up the question of model choice and examine three competing approaches. The first approach, reflected in the popular student growth percentiles (SGPs)…

This study examined the practical problem of covariate selection in propensity scores (PSs) given a predetermined set of covariates. Because the bias reduction capacity of a confounding covariate is proportional to the concurrent relationships it has with the outcome and treatment, particular focus is set on how we might approximate…

Through simulated data, five multidimensional computerized adaptive testing (MCAT) selection procedures with varying test lengths are examined and compared using different stopping rules. Fixed item exposure rates are used for all the items, and the Priority Index (PI) method is used for the content constraints. Two stopping rules, standard error…

This presentation focuses on the quasi-experimental methods used to select comparison schools for an evaluation of a federal investing in innovation (i3) validation grant. The Northeast Tennessee College and Career Ready Consortium (NETCO) consists of 29 high schools participating in a five-year program to expand students' access to rigorous…

A new option that allows students to choose which of their test scores to send to colleges has generated renewed criticism of the College Board. College Board officials tout the option, called Score Choice, as a way to ease test taker anxiety. Some prominent admissions officials have publicly described Score Choice as a sales tactic that will…

Despite repeated calls for educators to get more instructional mileage out of the assessment data they have at hand, two deterrents typically stand in the way of most educators' effective use of test data. First, there's a missing "realization", and second, there's a missing "skill". Educators who possess both this realization and this skill will…

Credit scores have a profound impact on home purchasing power and mortgage pricing, yet little is known about how credit scores influence households' residential location decisions. This study estimates the effects of credit scores on residential sorting behavior using a novel mortgage industry data set combining household demographic, credit, and…

Selection bias is a problem for mathematics education researchers interested in using observational rather than experimental data to make causal inferences about the effects of different instructional methods in mathematics on student outcomes. Propensity score methods represent 1 approach to dealing with such selection bias. This article…

A Bayesian approach for simultaneous optimization of test-based decisions is presented using the example of a selection decision for a treatment followed by a mastery decision. A distinction is made between weak and strong rules, and conditions for monotonicity of optimal weak and strong rules are presented. (Author/SLD)