This study evaluates various scoring methods including number-correct scoring, IRT theta scoring, and hybrid scoring in terms of scale-score stability over time. A simulation study was conducted to examine the relative performance of five scoring methods in terms of preserving the first two moments of scale scores for a population in a chain of…

Increasing use of item pools in large-scale educational assessments calls for an appropriate scaling procedure to achieve a common metric among field-tested items. The present study examines scaling procedures for developing a new item pool under a spiraled block linking design. The three scaling procedures are considered: (a) concurrent…

Large-scale international assessments rely on indicators of the resources that students report having in their homes to capture the financial capital of their families. The scaling methodology currently used to develop the Programme for International Student Assessment (PISA) background indices is designed to maximize within-country comparability…

This study examines the use of cross-classified random effects models (CCrem) and cross-classified multiple membership random effects models (CCMMrem) to model rater bias and estimate teacher effectiveness. Effect estimates are compared using CTT versus item response theory (IRT) scaling methods and three models (i.e., conventional multilevel…

This study compared vertical scaling results for the Rasch model from BILOG-MG and WINSTEPS. The item and ability parameters for the simulated vocabulary tests were scaled across 11 grades; kindergarten through 10th. Data were based on real data and were simulated under normal and skewed distribution assumptions. WINSTEPS and BILOG-MG were each…

Recently, nonparametric methods have been proposed that provide a dimensionally based description of test structure for tests with dichotomous items. Because such methods are based on different notions of dimensionality than are assumed when using a psychometric model, it remains unclear whether these procedures might lead to a different…

Empirical data analysis using the Mokken models of monotone homogeneity and double monotonicity is discussed. Results from the Mokken approach with 3 data sets (for a total of 425 elementary school students) pertaining to transitive interference items are compared to Rasch analysis. (SLD)

Examined convergent and discriminant validity of various self-efficacy measures across two studies, one involving 358 U.S. high school students and another involving 235 Korean female high school students. Across the studies the first-order confirmatory factor analyses provide support for both convergent validity of different self-efficacy…

A procedure for interpreting multiple-discrimination indices from a multidimensional item-response theory analysis is described and demonstrated with responses of 1,635 high school students to a multiple-choice test. The procedure consists of converting discrimination parameter estimates to direction cosines and analyzing the angular distances…