A probabilistic item response theory (IRT) model is developed for pair-comparison design in which the unfolding principle governing the choice process uses a discriminant process analogous to Thurstone's Law of Comparative Judgment. A simulation study demonstrates the feasibility of estimation, and two examples illustrate the implications for…

Examines subjects' processing in misleading test items. Suggests that the M-power for children, when assessed behaviorally, may increase with age in a discrete manner, and have the potential to generate interval scales of measurement. In addition, suggests that, in light of the results, what statisticians often consider "error of…

The application of generalizability theory to the reliability and error variance estimation for performance assessment scores is discussed. Decision makers concerned with performance assessment need to realize the restrictions that limit generalizability such as limitations that lead to reductions in the number of tasks possible, rater quality,…

The framework and procedures of generalizability theory are introduced and illustrated in this instructional module that uses a hypothetical scenario involving writing proficiency. Generalizability analyses are useful for understanding the relative importance of various sources of error and for designing efficient measurement procedures. (SLD)

Presents a general framework for designing sampling programs that ensure cost effectiveness, and managed errors kept within known and acceptable limits. (LZ)

Using structural equation technique to replicate results of Hasenfeld and Rafferty's causal model predicting public attitudes toward welfare state programs with the social ideologies of work ethic and social rights. By incorporating estimates of measurement error, results failed to support the authors' original conclusions. Operationalizing key…

A confirmatory factor analysis of 61 published data sets from multitrait-multimethod studies provides estimates of the trait, method, and error variances of measures used in the social and behavioral sciences. Traits accounted for less than 50 percent of the variance among construct measures overall. Implications for testing are discussed. (SLD)

Replenishing item pools for on-line ability testing requires innovative and efficient data collection. A method is proposed to collect test item calibration data in an on-line testing environment sequentially using locally D-optimum designs, thereby achieving high Fisher information for the item parameters. (SLD)

Recent within-twin estimates of schooling returns are considerably higher than existing estimates. This paper shows that small ability differences among twins can yield more upward omitted-ability bias (and more upward bias overall) in the instrumental variables estimate correcting for measurement error than in the standard within-twin estimate.…

American educators are participating in a harmful, unwinnable contest--the score-boosting game. Standardized tests yield an inaccurate picture of school staff's instructional practices. Administrators should provide assessment-literacy training, brief educational policymakers, encourage autonomous parent groups, analyze test items, and create…

One of the most important effect sizes used in meta-analysis is the standardized mean difference (SMD). In this article, the conditions under which SMD effect sizes based on different measures of the same construct are directly comparable are investigated. The results show that SMD effect sizes from different measures of the same construct are…

The term "gifted" implies a permanent superiority. However, the majority of children who score in the top few percentiles on ability and achievement tests in 1 grade do not retain their status for more than a year or 2. The tendency of those with high scores on one occasion to obtain somewhat lower scores on a later occasion is one…

This study investigates item parameter recovery, standard error estimates, and fit statistics yielded by the WINSTEPS program under the Rasch model and the rating scale model through Monte Carlo simulations. The independent variables were item response model, test length, and sample size. WINSTEPS yielded practically unbiased estimates for the…

As item response theory (IRT) becomes popular in educational and psychological testing, there is a need of reporting IRT-based effect size measures. In this study, we show how the standardized mean difference can be generalized into such a measure. A disattenuation procedure based on the IRT test reliability is proposed to correct the attenuation…

Intraindividual variability in positive and negative affect was assessed by the positive affect (Contentment, Joy, Vigor, Love, and Excitement) and negative affect (Depression, Hostility, Anxiety, Agitation, and Social Anxiety) subscales of the state version of the Comprehensive Personality and Affect Scales (COPAS) during a 3-week period. Using…