Exact testing was evaluated as a method for conducting Mantel-Haenszel differential item functioning (DIF) analyses with relatively small samples. A series of computer simulations found that the asymptotic Mantel-Haenszel and the exact method yielded very similar results across sample size, levels of DIF, and data sets. (SLD)

The potential for bias in reliability estimation and for errors in item selection when alpha or unit-weighted omega coefficients are used is explored under simulated conditions. Results suggest that composite reliability may be an assessment tool but should not be an item selection tool in structural equations. (SLD)

The effect of the conditioning score on the results of differential item functioning (DIF) analysis was examined with simulated data. The study demonstrates that results of DIF that rely on a conditioning score can be quite different depending on the conditioning variable that is selected. (SLD)

A historical perspective is provided of the concept of invariance in measurement theory, describing sample-invariant item calibration and item-invariant measurement of individuals. Invariance as a key measurement concept is illustrated through the measurement theories of E. L. Thorndike, L. L. Thurstone, and G. Rasch. (SLD)

Presents critical analysis of a paper "Testing Reading Comprehension Skills, Part One," in which the consideration concerns the inadequacy of taxonomies of skills to describe individual readers' processes and, hence, their usefulness in test construction. (15 references) (GLR)

How item bias indexes based on item response theory (IRT) identify bias that results from multidimensionality is demonstrated. Simulation results suggest that IRT-based bias indexes detect multidimensional items with bias but do not detect multidimensional items without bias. They also do not confound between-group differences on the primary test.…

The concept of information functions developed for dichotomous item response models is adapted for the partial credit model, and the information function is used to investigate collapsing and recoding categories of polytomously scored items from the National Assessment of Educational Progress. (SLD)

Predictive validity, generally taken to be the prime validity that occupationally normed interest inventories should demonstrate, is dependent on the capacity of an instrument to differentiate between occupations. A comparison of two methods of differentiation shows that a method using proportions of each occupational group to assign item-scoring…

A set of conditions is presented for the validity of inference for Item Response Theory (IRT) models applied to data collected from examinations that allow students to choose a subset of items. Common low-dimensional IRT models estimated by standard methods do not resolve the difficult problems posed by choice-based data. (SLD)

A technique is presented for applying the Rule Space methodology of cognitive diagnosis to assessment in a semantically rich domain. The approach bases diagnosis on item characteristics that are more abstract than individual problem-solving steps. The method is illustrated through a test of architectural knowledge completed by 122 architects. (SLD)

Explored how allowing examinees to select test questions affected examinee performance and test characteristics for a measure of ability to generate hypotheses about a situation. Results with 2,429 examinees who elected the choice condition on the Graduate Record Examination suggest that items are differentially attractive to examinees. (SLD)

Confirmatory factor analysis of data from 1,346 respondents to the Attributional Style Questionnaire (ASQ) (C. Peterson and others, 1982) reveals that adequate fit is provided by a three-factor attributional style model that includes context-dependent item sets. Results suggest that there is no such thing as a nonsituational attributional style.…

Examined a set of working memory scales containing two versions of test items that are reading and mathematics based. Data from 201 undergraduates support the hypothesis that an oblique two-factor model in which the factors are based on item content would fit the data well. (SLD)

Investigated the performance of the Poly-DIMTEST (PD) procedure (and associated computer program) in assessing the unidimensionality of test data produced by polytomous items through Monte Carlo simulation. Results show that PD can confirm unidimensionality for unidimensional simulated data and can detect lack of unidimensionality. (SLD)

Examines the relationship between Cronbach's coefficient alpha and the reliability of a composite of a prespecified set of interrelated nonhomogeneous components through simulation. Shows that alpha can over- or underestimate scale reliability at the population level. Illustrates the bias in terms of structural parameters. (SLD)