Item analysis is an integral part of operational test development and is typically conducted within two popular statistical frameworks: classical test theory (CTT) and item response theory (IRT). In this digital ITEMS module, Hanwook Yoo and Ronald K. Hambleton provide an accessible overview of operational item analysis approaches within these…

Differential item functioning (DIF) analyses are a routine part of the development of large-scale assessments. Less common are studies to understand the potential sources of DIF. The goals of this study were (a) to identify gender DIF in a large-scale science assessment and (b) to look for trends in the DIF and non-DIF items due to content,…

Essential for an effective criterion-referenced testing program is a set of test items that are "valid" indicators of the objectives they have been designed to measure. Unfortunately, the complex matter of assessing item validity has received only limited attention from educational measurement specialists. One promising approach to the item…

The purpose of this paper is to describe some of the current changes in test development that are taking place because of the availability and capabilities of computers, especially microcomputers. Item banking and test assembly are discussed, and a comprehensive testing system is described. (BW)

The agreement between item response theory-based and Mantel Haenszel (MH) methods in identifying biased items on tests was studied. Data came from item responses of four spaced samples of 1,000 examinees each--two samples of 1,000 Anglo-American and two samples of 1,000 Native American students taking the New Mexico High School Proficiency…

Although item bias statistics are widely recommended for use in test development and test analysis work, problems arise in their interpretation. The purpose of the present research was to evaluate the validity of logistic test models and computer simulation methods for providing a frame of reference for item bias statistic interpretations.…

The purpose of the present research was to study, systematically, the "goodness-of-fit" of the one-, two-, and three-parameter logistic models. We studied, using computer-simulated test data, the effects of four variables: variation in item discrimination parameters, the average value of the pseudo-chance level parameters, test length,…

Latent trait models may offer considerable potential for the improvement of educational measurement practices, but until recently, they have received only limited attention from measurement specialists. This paper provides a brief introduction to latent trait models, and provides test practitioners with a non-technical introduction to the…

The study compared two promising item response theory (IRT) item-selection methods, optimal and content-optimal, with two non-IRT item selection methods, random and classical, for use in fixed-length certification exams. The four methods were used to construct 20-item exams from a pool of approximately 250 items taken from a 1985 certification…

Topics concerning latent trait theory are addressed: (1) dimensionality of latent space, local independence, and item characteristic curves; (2) models--equations, parameter estimation, testing assumptions, and goodness of fit, (3) applications test developments, item bias, tailored testing and equating; and (4) advantages over classical…

Issues in preparing a review form to detect item bias in tests are discussed and the first draft of an item bias review form is presented. While stereotyping is the consistent representation of a given group in a particular light, bias is the presence of some characteristic of an item that results in differential performance of two individuals of…

The success of objectives-based programs depends to a considerable extent on how effectively students and teachers assess mastery of objectives and make decisions for future instruction. While educators disagree on the usefulness of criterion-referenced tests the position taken in this monograph is that criterion-referenced tests are useful, and…

Item response model applications to National Assessment of Educational Progress (NAEP) data specifically aimed at the uses of item response models in mathematics item banking are discussed. Approaches for addressing goodness of fit were organized into three categories: Checks on model assumptions, expected features, and additional model…

This paper highlights some special characteristics of criterion-referenced tests while comparing them with norm-referenced tests. Psychometric considerations involved in constructing a criterion-referenced test, including item analysis, reliability, and validity, are discussed, along with application of criterion-referenced testing to individual…