A conceptualization of multiple-choice exams in terms of signal detection theory (SDT) leads to simple measures of item difficulty and item discrimination that are closely related to, but also distinct from, those used in classical item analysis (CIA). The theory defines a "true split," depending on whether or not examinees know an item,…

For proper assessment selection understanding the statistical similarities amongst assessments that measure the same, or very similar, topics is imperative. This study seeks to extend the comparative analysis between the brief electricity and magnetism assessment (BEMA) and the conceptual survey of electricity and magnetism (CSEM) presented by…

Purpose: This study examined the quality of English test items using psychometric and linguistic characteristics among Grade Six pupils. Method: Contrary to the conventional approach of relying only on statistics when investigating item quality, this study adopted a mixed-method approach by employing psychometric analysis and cognitive interviews.…

The aim of this study was to develop, for the benefit of both test takers and test score users, enhanced "TOEFL ITP"® test score reports that go beyond the simple numerical scores that are currently reported. To do so, we applied traditional scale anchoring (proficiency scaling) to item difficulty data in order to develop performance…

This study was inspired by the final regulations for the No Child Left Behind Act (NCLB) indicating that each state has the option to develop a new assessment for students whose disabilities have kept them from obtaining proficiency. Sets of high school science achievement items were enhanced for the new test. A 3-by-2, within subjects,…

The present study examined the technical adequacy of curriculum-based measures (CBMs) of early numeracy. Six 1-min early mathematics tasks were administered to 137 kindergarten and first-grade students, along with an omnibus test of early mathematics. The CBM measures included Count Out Loud, Quantity Discrimination, Number Identification, Missing…

The purpose of this study was to develop general outcome measures (GOM) in mathematics so that teachers could focus their instruction on needed prerequisite skills. We describe in detail, the manner in which content-related evidence was established and then present a number of statistical analyses conducted to evaluate the technical adequacy of…

The purpose of this paper is to present a generalization of the concept of item difficulty to test items that measure more than one dimension. Three common definitions of item difficulty were considered: the proportion of correct responses for a group of individuals; the probability of a correct response to an item for a specific person; and the…

Cliff's Index "c" derived from an item dominance matrix is utilized in a clustering approach, termed extracting Reliable Guttman Orders (ERGO), to isolate Guttman-type item hierarchies. A comparison of factor analysis to the ERGO is made on social distance data involving multiple ethnic groups. (Author/BW)

Violations of item unidimensionality on language tests produced distorted estimates of person ability, and violations of person unidimensionality produced distorted estimates of item difficulty. The Bejar Method was sensitive to such distortions. (Author)

A test score can be used for individual instructional diagnosis after determining whether: (1) difficulty of the test items was consistent with the complexity of the content measured; (2) items measuring the same underlying process were about equally difficult; and (3) partial credit scoring would increase the reliability of the diagnosis. (LMO)

The purpose of the present research was to study, systematically, the "goodness-of-fit" of the one-, two-, and three-parameter logistic models. We studied, using computer-simulated test data, the effects of four variables: variation in item discrimination parameters, the average value of the pseudo-chance level parameters, test length,…

Under given conditions, conventional testing and computer-generated repeatable testing (CGRT) are equally effective for estimating examinee ability; CGRT is more effective for estimating the mean ability level of a group and less effective for estimating ability differences among individuals. These conclusion are drawn from domain-referenced test…

This article discusses the application of the Golden Rule procedure to items of the Scholastic Aptitude Test. Using item response theory, the analyses indicate that the Golden Rule procedures are ineffective in detecting biased items and may undermine the reliability and validity of tests. (Author/JAZ)

The purpose of this research was to reinvestigate the accuracy of three item bias detection procedures: (1) Linn and Harnisch's pseudo-IRT(Z) method; (2) Camilli's chi-square technique; and (3) Angoff's revised transformed item difficulty method. These methods are applied when the minority group sample size is too small to obtain stable estimates…