It is desirable for an educational assessment to be constructed of items that can differentiate different performance levels of test takers, and thus it is important to estimate accurately the item discrimination parameters in either classical test theory or item response theory. It is particularly challenging to do so when the sample sizes are…

Reporting standard errors of equating has been advocated as a standard practice when conducting test equating. The two most widely applied procedures for standard errors of equating including the bootstrap method and the delta method are either computationally intensive or confined to the derivations of complicated formulas. In the current study,…

This study presents new models for item response functions (IRFs) in the framework of the D-scoring method (DSM) that is gaining attention in the field of educational and psychological measurement and largescale assessments. In a previous work on DSM, the IRFs of binary items were estimated using a logistic regression model (LRM). However, the LRM…

The frequently neglected and often misunderstood relationship between classical test theory and item response theory is discussed for the unidimensional case with binary measures and no guessing. It is pointed out that popular item response models can be directly obtained from classical test theory-based models by accounting for the discrete…

The purpose of this study was to examine the impact of dimensionality, common-item set format, and different scale linking methods on preserving equity property with mixed-format test equating. Item response theory (IRT) true-score equating (TSE) and IRT observed-score equating (OSE) methods were used under common-item nonequivalent groups design.…

Examines the question of how large a sample must be in order to produce empirical Bayes estimates which are preferable to other commonly used estimates, such as proportion correct observed score. (Author/RC)

Numerous studies have shown increasing item reliabilities as an effect of the item position in personality scales. Traditionally, these context effects are analyzed based on item-total correlations. This approach neglects that trends in item reliabilities can be caused either by an increase in true score variance or by a decrease in error…

Author supports the credibility of the propositions that: (1) the true component of a score is proportional to the number of equivalent elements that contribute to it. And, (2) the error component of a score is proportional to the square root of the number of equivalent elements that contribute to it. (Author/MB)

The current research was conducted to investigate the validity of automated essay scoring (AES) by comparing group mean scores assigned by an AES tool, IntelliMetric [TM] and human raters. Data collection included administering the Texas version of the WriterPlacer "Plus" test and obtaining scores assigned by IntelliMetric [TM] and by…

The test-retest reliability of one single dichotomous item is discussed. Various indices are derived for summarizing the stability of a dichotomy, based on the concept of Platonic true scores. Both open-ended and multiple choice items are considered. (Author/JKS)

For the six available sets of empirical data, the discrimination (slope) parameter of the logistic item characteristic curve was found to have a significant positive correlation over items with the difficulty (location) parameter. This unpleasant situation can be eliminated by a suitably chosen transformation of the ability scale. (Author/RC)

One of the most important effect sizes used in meta-analysis is the standardized mean difference (SMD). In this article, the conditions under which SMD effect sizes based on different measures of the same construct are directly comparable are investigated. The results show that SMD effect sizes from different measures of the same construct are…

This handbook treats a restricted set of statistical procedures for addressing some of the most prevalent technical issues that arise in domain-referenced testing. The procedures discussed here were chosen because they do not necessitate extensive computations. The five major sections of the paper cover: (1) item analysis procedures for using data…

Given known item parameters for a test, unbiased estimators are derived for an examinee's ability parameter, his or her proportion correct true score for the variances of these parameters and for the parallel forms reliability of the maximum likelihood estimator of the ability parameter. (Author/JKS)

The performance of three equating methods--the presmoothed equipercentile method, the item response theory (IRT) true score method, and the IRT observed score method--were examined based on three equating criteria: the same distributions property, the first-order equity property, and the second-order equity property. The magnitude of the…