This article describes how a Brier score analysis can be used as an evaluative tool to estimate the predictive accuracy of a course placement policy that was established based on professional or subjective judgment. The policy being evaluated uses the score received on the mathematics portion of the SAT examination as the primary mechanism to…

In this study, differential item functioning (DIF) methods utilizing 14 different matching variables were applied to assess DIF in the constructed-response (CR) items from 6 forms of 3 mixed-format tests. Results suggested that the methods might produce distinct patterns of DIF results for different tests and testing programs, in that the DIF…

The purpose of the study is to present research focused on validating the four algebra cognitive models in Gierl, Wang, et al., using student response data collected with protocol analysis methods to evaluate the knowledge structures and processing skills used by a sample of SAT test takers.

Two methods of computing test equating coefficients under item response theory by the following authors are compared: (1) B. H. Loyd and H. D. Hoover (1980); and (2) M. L. Stocking and F. M. Lord (1983). Conditions under which the method of Stocking and Lord is preferable are described. (SLD)

Concepts behind plausible values in estimating population characteristics from sparse matrix samples of item responses are discussed. The use of marginal analyses is described in the context of the National Assessment of Educational Progress, and the approach is illustrated with Scholastic Aptitude Test data for 9,075 high school seniors. (SLD)

The construct validity of algebra word problems for measuring quantitative reasoning was examined, focusing on an analysis of problem attributes and on the analysis of constructed-response solutions. Constructed-response solutions to 20 problems from the Graduate Record Examinations (GRE) General Test were collected from 51 undergraduates.…

Calculating the reliability of a testlet-based test is demonstrated using data from 1,812 males and 2,216 females taking the Scholastic Aptitude Test verbal section and 3,866 examinees taking another reading test. Traditional reliabilities calculated on reading comprehension tests constructed of four testlets provided substantial overestimates.…

The effectiveness of the freshman admissions prediction equation (FAPE) at the University of Georgia (Athens) was studied, using data for 3,378 freshman (1,490 males and 1,888 females) in 1987-88. For the 1987-88 data, a previous (1982) prediction equation functioned poorly in terms of predictive accuracy. New prediction models were constructed…

Analyses for results of a series of studies examining intercorrelations among a set of as many as p+1 variables are presented. Several estimators of a pooled or average correlation vector and its variances are derived for cases in which some studies do not report complete correlation matrices. A test of the homogeneity (consistency) of the…

A method was developed for identifying students who may be at high risk for failing the State Board Licensing Examination (SBE) for registered nurses. The subjects used in developing prediction equations included 50 students who graduated from the nursing program at Saint Joseph's College (SJC) in North Windham (Maine) during the years 1983-84.…

Research efforts have established that the Mantel-Haenszel procedure (MHP) is an effective method for detecting the presence of test items exhibiting differential item functioning (DIF). While the MHP has been advocated for situations where item response theory based methods may not be usable, recent findings have suggested that the performance of…