This research applied a cognitive model to identify item features that lead to irrelevant variance on the Test of Spoken English[TM] (TSE[R]). The TSE is an assessment of English oral proficiency and includes an item that elicits a description of a statistical graph. This item type sometimes appears to tap graph-reading skills--an irrelevant…

This paper examines item response theory (IRT) scale transformations and IRT scale linking methods used in the Non-Equivalent Groups with Anchor Test (NEAT) design to equate two tests, X and Y. It proposes a unifying approach to the commonly used IRT linking methods: mean-mean, mean-var linking, concurrent calibration, Stocking and Lord and…

This report, the fifth annual report on the Colorado Student Assessment Program (CSAP), provides policymakers, educators, parents, and the community with a general accounting and precise overview of the performance of Colorado's third, fourth, fifth, sixth, seventh, eighth, ninth, and tenth grade students relative to the Colorado State Model…

Statistical procedures for detecting differential item functioning (DIF) are often used to screen items for construct irrelevant variance. Standard DIF detection procedures focus on only one categorical variables at an aggregated group or one-way level, like gender or ethnicity/race. Building on previous work by P. Hu and N. Dorans (1998), N.…

The delta-plot method is used to identify which common items in a common item nonequivalent groups design for test equating show large changes in their p-values across administrations. Outliers in that plot denote differential item behavior and are candidates for exclusion from the common item pool. This study investigated whether keeping or…

The purpose of this report is to inform users and other interested parties about the development, content and technical characteristics of the Virginia Standards of Learning (SOL) assessments. It provides information for the 2003 "SOL" cycle that comprises the fall 2003 and spring 2004 administrations. The report is divided into three…

As a continuing part of its validity studies, the Association of American Medical Colleges commissioned a study of the speediness of the Medical College Admission Test (MCAT). If speed is a hidden part of the test, it is a threat to its construct validity. As a general rule, the criterion used to indicate lack of speediness is that 80% of the…

This study extended prior Law School Admission Council (LSAC) research related to the item response theory (IRT) local item independence assumption into the realm of classical test theory. Initially, results from the Law School Admission Test (LSAT) and two other tests were investigated to determine the approximate state of local item independence…

This study investigated the practical effects of local item dependence (LID) on item response theory (IRT) true-score equating. A scenario was defined that emulated the Law School Admission Test (LSAT) preequating model, and data were generated to assess the impact of different degrees of LID on final equating outcomes. An extreme amount of LID…

This study explored the impact of various degrees of violations of the item response theory (IRT) local independence assumption on the Law School Admission Test (LSAT) calibration and score distribution estimates. Initially, results from the LSAT and two other tests were investigated to determine the approximate state of local item dependence…

Two new indices to detect answer copying on a multiple-choice test, S(1) and S(2) (subscripts), are proposed. The S(1) index is similar to the K-index (P. Holland, 1996) and the K-overscore(2), (K2) index (L. Sotaridona and R. Meijer, in press), but the distribution of the number of matching incorrect answers of the source (examinee s) and the…

The effects of skewed theta distributions on indices of differential item functioning (DIF) were studied, comparing Mantel Haenszel (N. Mantel and W. Haenszel, 1959) and DFIT (N. S. Raju, W. J. van der Linden, and P. F. Fleer) (noncompensatory DIF). The significance of the study is that in educational and psychological data, the distributions one…

Item response theory models arose from the inherent limitations of classical test theory methods of test analysis. A brief description of those limitations and the corresponding enhancements provided by item response models is provided. Further, an examination of the popular Rasch one-parameter latent trait model is undertaken. Specific…

The purpose of this study was to determine the extent and magnitude of differential item functioning (DIF) between males and females in existing attitudinal data sets. The focus was on the approximate proportion of items that show statistically significant DIF in selected data sets concerning attitude scales, the magnitude of this DIF, and whether…

The types and frequency of markings students made on multiple choice tests were studied as an indication of students' metacognitive test-taking strategies. Also studied was whether instruction in test-taking strategies affected students' fluency and flexibility of strategy use as demonstrated through the marks they made on tests. Participants were…