Psychometric theory has been one of psychology's stronger foundations and a major contributor to the recognition of psychology as a scientific discipline. Basic principles of psychometric theory led to the development of respected intelligence tests and large and comprehensive testing and assessment programs. This paper synthesizes major…

LISREL-type structural equation modeling is a powerful statistical technique that seems appropriate for social science variables which are complex and difficult to measure. The literature on the specification, estimation, and testing of such models is voluminous. The greatest proportion of this literature, however, focuses on the technical aspects…

Four methods for determining the dimensionality of a set of test items were compared: (1) linear factor analysis; (2) residual analysis; (3) nonlinear factor analysis; and (4) Bejar's method. Five artificial test data sets (for 40 items and 1500 examinees) were generated, consistent with the three-parameter logistic model and the assumption of…

This paper discusses distribution theory and power computations for four common "tests of combined significance." These tests are calculated using one-sided sample probabilities or p values from independent studies (or hypothesis tests), and provide an overall significance level for the series of results. Noncentral asymptotic sampling…

The effects of a multidimensional latent trait space on estimation of item and person parameters by the computer program LOGIST are examined. Several item pools were simulated that ranged from truly unidimensional to an inconsequential general latent trait. Item pools with intermediate levels of prepotency of the general latent trait were also…

Latent trait models are presented that can be used for test design in the context of a theory about the variables that underlie task performance. Examples of methods for decomposing and testing hypotheses about the theoretical variables in task performance are given. The methods can be used to determine the processing components that are involved…

The effect of changes in the achievement level of black high school students on their chances of attending postsecondary schools was assessed. Data from the High School and Beyond survey 1980 sophomore cohort were used to estimate the probability of attending two-year and four-year colleges for blacks and whites, given changes in black achievement…

This paper analyzes the quantitative and situational structure of algebra story problems, uses these materials to propose an interpretive framework for written problem soving protocols, and then presents an exploratory study of the episodic structure of algebra story problem solving in a sizable group of mathematically competent subjects. Analyses…

Many multidimensional item response theory (IRT) models have been proposed. A comparison is made between the so-called full information models and the models that use only pairwise information. Three multidimensional models described are: (1) the compensatory model of R. D. Bock and M. Aitken (1981) using the computer program TESTFACT; (2) a model…

Four discussions of test construction based on item response theory (IRT) are presented. The first discussion, "Test Design as Model Building in Mathematical Programming" (T. J. J. M. Theunissen), presents test design as a decision process under certainty. A natural way of modeling this process leads to mathematical programming. General…

This document contains three papers from the Methodology Project of the Center for the Study of Evaluation. Methods for characterizing test accuracy are reported in the first two papers. "Bounds on the K Out of N Reliability of a Test, and an Exact Test for Hierarchically Related Items" describes and illustrates how an extension of a…

The current state of usage of regression models in analysis of variance (ANOVA) designs is empirically examined, and examples of several statistical errors made in usage are presented. The assumptions of the general linear model are that all predictors are known without error of measurement and are fixed with no replication or sample variation; in…

The efficacy of the CATES system for making training decisions and determining student proficiency in Naval in-flight training proposed in an earlier study (Rankin and McDaniel, 1980) is compared with the present system of instructor judgments for performance assessment. The current study used 29 newly-designated naval aviators undergoing Fleet…

The psychometric properties associated with the "specific objectivity" of the simple logistic model are acknowledged to be far superior to those of competing models. It is therefore the ideal model to be used for test development or for the analysis of any measuring instrument developed within a well-defined psychological construct. In…

The results of six major projects are discussed including a comprehensive mathematical and statistical analysis of the problems caused by errors of measurement in linear models for assessing change. In a general matrix representation of the problem, several new analytic results are proved concerning the parameters which affect bias in…