Researchers often assess causal effects of educational programs or policies using educational assessment data. This dissertation explores novel methods of estimating causal effects in educational assessment data and is broken into three parts. The first part proposes a regression discontinuity design with an ordinal running variable to assess the…

It is a much-lamented fact that research with the potential to inform or influence education policy instead remains policy inert. There are many reasons for this frustrating state of affairs, including a lack of strategic thinking on the part of researchers on how to successfully accomplish outreach--as opposed to communication with peers…

Our purpose is to proffer QuantCrit methodological approaches to interrogate notions of statistical practice by convention. We present two approaches to meta-analysis and mean effect size calculations for student achievement. The first approach is the conventional approach which applies between-group differences to calculate effect sizes…

We used the Stanford education data archive (SEDA) data to examine the heterogeneity among urban school districts in the United States. The SEDA 2.1 includes data sets on students' mathematics (Math) and English language arts (ELA) achievement from 2008 to 2014 at the district level. Growth mixture modeling was used to uncover the underlying…

In large-scale educational assessment data consistent with a simple-structure multidimensional item response theory (MIRT) model, where every item measures only one latent variable, separate unidimensional item response theory (UIRT) models for each latent variable are often calibrated for practical reasons. While this approach can be valid for…

When an assessment undergoes changes to the administration or instrument, bridge studies are typically used to try to ensure comparability of scores before and after the change. Among the most common and powerful is the common population linking design, with the use of a linear transformation to link scores to the metric of the original…

The NAEP EDM Competition required participants to predict efficient test-taking behavior based on log data. This paper describes our top-down approach for engineering features by means of psychometric modeling, aiming at machine learning for the predictive classification task. For feature engineering, we employed, among others, the Log-Normal…

The design and psychometric methodology of the National Assessment of Educational Progress (NAEP) is constantly evolving to meet the changing interests and demands stemming from a rapidly shifting educational landscape. NAEP has been built on strong research foundations that include conducting extensive evaluations and comparisons before new…

The present study examines the features and quality of visual representations (VRs) created by middle school students with learning disabilities and difficulties in mathematics in response to a released fraction item from the National Assessment of Educational Progress (NAEP). Relations between VR quality and scores on other measures of fraction…

In high-stakes, large-scale, standardized tests with certain time limits, examinees are likely to engage in either one of the three types of behavior (e.g., van der Linden & Guo, 2008; Wang & Xu, 2015): solution behavior, rapid guessing behavior, and cheating behavior. Oftentimes examinees do not always solve all items due to various…

This study investigates the relationship between text-to-speech (TTS) usage and item-by-item performance in the 2017 eighth-grade National Assessment of Educational Progress (NAEP) math assessment, focusing on students with disabilities (SWDs), English language learners (ELLs), and their general education (GE) peers. Results indicate that all…

Chetty et al. (2022) say county density of cross-class friendships (referred to here as "adult-bridging capital") has causal impacts on social mobility within the United States. We instead find that social mobility rates are a function of county density of family capital (higher marriage rates and two-person households), community…

The purpose of this APA BEA Racial Disparities Task Force report is to examine the role of racism and bias on educational opportunity and achievement disparities experienced by children. Specifically, we seek to link racism explicitly to opportunity gaps by examining how racism operates on multiple levels. Using critical race theory,…

In the absence of clear incentives, achievement tests may be subject to the effect of slipping where item response functions have upper asymptotes below one. Slipping reduces score precision for higher latent scores and distorts test developers' understandings of item and test information. A multidimensional four-parameter normal ogive model was…

To ensure the statistical result validity, model-data fit must be evaluated for each item. In practice, certain actions or treatments are needed for misfit items. If all misfit items are treated, much item information would be lost during calibration. On the other hand, if only severely misfit items are treated, the inclusion of misfit items may…