In item response theory (IRT), preequating depends upon item parameter estimate invariance. Three separate simulations, all using the unidimensional three-parameter logistic item response model, were conducted to study the impact of the following variables on preequating: (1) mean differences in ability; (2) multidimensionality in the data; and…

This was a replicative study that was initiated with a journeyman level certification instrument for an international union, when industry monitors were observed suggesting to examinees to "go with your first response." The question arose whether this was a researched-based practice. If not, wouldn't this practice inject constant error…

The Test of Standard Written English (TSWE) is a 50-item multiple choice instrument designed to assess the ability of college students to use English. In this study, based upon a sample of 45 students, the TSWE was revalidated with writing samples. The coefficient of 0.54 was most impressive given that the TSWE scores were restricted to those…

LISREL-type structural equation modeling is a powerful statistical technique that seems appropriate for social science variables which are complex and difficult to measure. The literature on the specification, estimation, and testing of such models is voluminous. The greatest proportion of this literature, however, focuses on the technical aspects…

This report presents the rationale, development, and standardization of the Air Force Officer Qualifying Test (AFOQT) Form O. The test is used to select individuals for officer commissioning programs, and candidates for pilot and navigator training. Form O contains 380 items organized in 16 subtests. All items are administered in a single test…

The purpose of this paper is to present two new alternative methods to the current goodness of fit methodology. With the increase use of computerized adaptive test (CAT), the ability to determine the accuracy of calibrated item parameter estimates is paramount. The first method applies a normalizing transformation to the logistic residuals to make…

In order to evaluate standard setting procedures, apart from the more commonly applied approach of simply comparing the derived standards or failure rates across various techniques, this study investigated the errors of classification associated with the contrasting groups procedures. Monte Carlo simulations were employed to produce…

The current state of usage of regression models in analysis of variance (ANOVA) designs is empirically examined, and examples of several statistical errors made in usage are presented. The assumptions of the general linear model are that all predictors are known without error of measurement and are fixed with no replication or sample variation; in…

The results of six major projects are discussed including a comprehensive mathematical and statistical analysis of the problems caused by errors of measurement in linear models for assessing change. In a general matrix representation of the problem, several new analytic results are proved concerning the parameters which affect bias in…

When criterion-referenced tests are used to assign examinees to states reflecting their performance level on a test, the better known methods for determining test length, which consider relationships among domain scores and errors of measurement, have their limitations. The purpose of this paper is to present a computer system named TESTLEN, which…

Multiple matrix sampling of three subscales of the California Psychological Inventory was used to investigate the effects of four variables on error estimates of the mean (EEM) and variance (EEV). The four variables were examinee population size (600, 450, 300, 150, 100, and 75); number of subtests, (2, 3, 4, 5, 6, and 7), hence the number of…

Using data from High School and Beyond, this study empirically investigated the extent of within-variable, between-occasion error covariances among variables included in educational achievement models. Little evidence was found to support the statement that reliability estimates for social background variables are inflated because of correlated…

Thirty-eight methods are presented for either setting standards or adjusting them based on an analysis of classification error rates. A trilevel classification scheme is used to categorize the methods, and 10 criteria of technical adequacy and practicability are proposed to evaluate them. (Author/LMO)

Reviews problems involved in analyzing data on educational or training needs and details use of the proportionate reduction in error (PRE) approach (Hildebrand, et al., 1977), which predicts the probability that certain combinations of a joint distribution will occur, then tests to see how closely the prediction matches the observation. (MBR)

The purpose of this study was to compare and evaluate three online pretest item calibration/scaling methods in terms of item parameter recovery when the item responses to the pretest items in the pool would be sparse. The three methods considered were the marginal maximum likelihood estimate with one EM cycle (OEM) method, the marginal maximum…