This paper introduces an approach that uses latent class analysis to identify cut scores (LCA-CS) and categorize respondents based on context scales derived from largescale assessments like PIRLS, TIMSS, and NAEP. Context scales use Likert scale items to measure latent constructs of interest and classify respondents into meaningful ordered…

The "2nd Annual WPI-UMASS-UPENN EDM Data Mining Challenge" required contestants to predict efficient testtaking based on log data. In this paper, we describe our theory-driven and psychometric modeling approach. For feature engineering, we employed the Log-Normal Response Time Model for estimating latent person speed, and the Generalized…

The NAEP EDM Competition required participants to predict efficient test-taking behavior based on log data. This paper describes our top-down approach for engineering features by means of psychometric modeling, aiming at machine learning for the predictive classification task. For feature engineering, we employed, among others, the Log-Normal…

This article explores the associations between the achievement of economically disadvantaged students and the presence of state policies that include student achievement in teacher evaluations. We looked at student achievement across all 50 states from 2007 through 2013. A simple comparison of states with and without the policy suggested that…

To ensure the statistical result validity, model-data fit must be evaluated for each item. In practice, certain actions or treatments are needed for misfit items. If all misfit items are treated, much item information would be lost during calibration. On the other hand, if only severely misfit items are treated, the inclusion of misfit items may…

The present study examined the ability of content domain (algebra, geometry, rational number, probability) to classify mathematics misconceptions. The study was conducted with 1,133 students in 53 algebra and geometry classes taught by 17 teachers from three high schools and one middle school across three school districts in a Midwestern state.…

Test scores are commonly reported in a small number of ordered categories. Examples of such reporting include state accountability testing, Advanced Placement tests, and English proficiency tests. This paper introduces and evaluates methods for estimating achievement gaps on a familiar standard-deviation-unit metric using data from these ordered…

Test scores are commonly reported in a small number of ordered categories. Examples of such reporting include state accountability testing, Advanced Placement tests, and English proficiency tests. This article introduces and evaluates methods for estimating achievement gaps on a familiar standard-deviation-unit metric using data from these ordered…

Large-scale assessment programs such as the National Assessment of Educational Progress (NAEP), Trends in International Mathematics and Science Study (TIMSS), and Programme for International Student Assessment (PISA) use a sophisticated assessment administration design called matrix sampling that minimizes the testing burden on individual…

This article describes a comprehensive approach to fully automated assessment of children's oral reading fluency (ORF), one of the most informative and frequently administered measures of children's reading ability. Speech recognition and machine learning techniques are described that model the 3 components of oral reading fluency: word accuracy,…

Background: Large-scale survey assessments have been used for decades to monitor what students know and can do. Such assessments aim at providing group-level scores for various populations, with little or no consequence to individual students for their test performance. Students' test-taking behaviors in survey assessments, particularly the level…

English Language Learners (ELLs) are among the fastest-growing demographic group in U.S. public schools. There are numerous recent reports documenting this phenomenon. Some reports estimate the numbers of ELLs enrolled in U.S. public schools, and other reports approximate the growth in ELL enrollment over the past five to ten years. Still, there…

Using eight waves of longitudinal data on a statewide kindergarten cohort of English learners (ELs), I examined ELs' tenure in language-learning programs and their academic performance following reclassification as fluent English proficient. I employed discrete-time survival analysis to estimate the average time to and grade of reclassification…

This article provides examples of how serious misconceptions can result when only "all student" scores from the National Assessment of Educational Progress (NAEP) are used for simplistic state-to-state comparisons. Suggestions for better treatment are presented. The article also compares Kentucky's eighth grade EXPLORE testing to NAEP…

The educational achievement and attainment of young people in the United States has been a long-standing issue of concern. While analyses of long-term trend data from the National Assessment of Educational Progress (NAEP) show that students in the United States have made gains in reading and mathematics over the past few decades, a sizeable…