We review the GIDNA and CDM packages in R for fitting cognitive diagnosis/diagnostic classification models. We first provide a summary of their core capabilities and then use both simulated and real data to compare their functionalities in practice. We found that the most relevant routines in the two packages appear to be more similar than…

Increasing use of item pools in large-scale educational assessments calls for an appropriate scaling procedure to achieve a common metric among field-tested items. The present study examines scaling procedures for developing a new item pool under a spiraled block linking design. The three scaling procedures are considered: (a) concurrent…

In recent guidelines for fair educational testing it is advised to check the validity of individual test scores through the use of person-fit statistics. For practitioners it is unclear on the basis of the existing literature which statistic to use. An overview of relatively simple existing nonparametric approaches to identify atypical response…

This simulation study compared the utility of various discrepancy measures within a posterior predictive model checking (PPMC) framework for detecting different types of data-model misfit in multidimensional Bayesian network (BN) models. The investigated conditions were motivated by an applied research program utilizing an operational complex…

Background: When studying student performance across different countries or cultures, an important aspect for comparisons is that of score comparability. In other words, it is imperative that the latent variable (i.e., construct of interest) is understood and measured equivalently across all participating groups or countries, if our inferences…

The purpose of this quantitative study is to compare a new assessment tool, the SkillsUSA Connect Assessment with the NOCTI assessment to determine which test results in more students achieving success. A quantitative study, designed to compare test scores of students taking the NOCTI assessment and new assessments from SkillsUSA, called the…

Test equating is a statistical procedure for adjusting for test form differences in difficulty in a standardized assessment. Equating results are supposed to hold for a specified target population (Kolen & Brennan, 2004; von Davier, Holland, & Thayer, 2004) and to be (relatively) independent of the subpopulations from the target population (see…

The statistical assessment of dimensionality provides evidence of the underlying constructs measured by a survey or test instrument. This study focuses on educational measurement, specifically tests comprised of items described as multidimensional. That is, items that require examinee proficiency in multiple content areas and/or multiple cognitive…

Measurement invariance is a common consideration in the evaluation of the validity and fairness of test scores when the tested population contains distinct groups of examinees, such as examinees receiving different forms of a translated test. Measurement invariance in polytomous items has traditionally been evaluated at the item-level,…

This study uses a multi-level multivariate propensity score matching approach to examine the synthetic cohort design (SCD) in estimating the schooling effect on mathematics proficiency of the focal cohort 2 (8th graders). Collecting 7th and 8th graders at the same time point, the SCD is sufficient in estimating the schooling effect under the…

This article demonstrates the use of a new class of model-free cumulative sum (CUSUM) statistics to detect person fit given the responses to a linear test. The fundamental statistic being accumulated is the likelihood ratio of two probabilities. The detection performance of this CUSUM scheme is compared to other model-free person-fit statistics…

The theoretical differences between the traditional definition of dimensionality and the more recently defined notion of essential dimensionality are presented. Monte Carlo simulations are used to demonstrate the utility of W. F. Stout's procedure to assess the essential unidimensionality of the latent space underlying a set of terms. The…

With the advent of No Child Left Behind (NCLB), the context for NAEP participation is changing. Whereas in the past participation in NAEP has always been voluntary, participation is now mandatory for some grade and subjects among schools receiving Title I funds. While this will certainly raise school-level participation rates in the mandated…

This document is designed to assist school district personnel in the identification of intervention strategies that have a good probability of increasing the district's mean score on the Goal IV Educational Quality Assessment (EQA) instrument. Appropriate educational research has been reviewed and distilled into seven propositions that are…

The Second International Conference on Educational Data Mining (EDM2009) was held at the University of Cordoba, Spain, on July 1-3, 2009. EDM brings together researchers from computer science, education, psychology, psychometrics, and statistics to analyze large data sets to answer educational research questions. The increase in instrumented…