A case is presented for adjusting the scores for free response items in the Advanced Placement (AP) tests. Using information about the rating process from the reliability studies, administrations of the AP test for three subject areas, psychology, computer science, and English language and composition, are analyzed. In the reliability studies, 299…

The relationship between levels of reliability and the power of two bias and differential item functioning (DIF) detection methods is examined. Both methods, the Mantel-Haenszel (MH) procedure of P. W. Holland and D. T. Thayer (1988) and the Simultaneous Item Bias (SIB) procedure of R. Shealy and W. Stout (1991), use examinees' raw scores as a…

Generalizability theory (g-theory) was used, as an alternative to classical test theory, to evaluate measurement error in a behaviorally anchored role-play measure, highlighting the usefulness of this theory in instrument development. G-theory partitions an observed score into the universe score and error scores associated with separate sources of…

This profile presents and summarizes available information about the quality of data from the five surveys that comprise the SASS, along with background material on the survey design and procedures for the following: (1) School Survey; (2) School Administrator Survey; (3) Teacher Demand and Shortage Survey; (4) Teacher Survey; and (5) Teacher…

The robustness of a partial credit (PC) model-based computerized adaptive test's (CAT's) ability estimation to items that did not fit the PC model was investigated. A CAT program was written based on the PC model. The program used maximum likelihood estimation of ability. Item selection was on the basis of information. The simulation terminated…

Rank ordering examinees is an easier task for judges than is awarding numerical ratings. A measurement model for rankings based on Rasch's objectivity axioms provides linear, sample-independent and judge-independent measures. Estimates of examinee measures are obtained from the data set of rankings, along with standard errors and fit statistics.…

The current status of item response theory (IRT) is discussed. Several IRT methods exist for assessing whether an item is biased. Focus is on methods proposed by L. M. Rudner (1975), F. M. Lord (1977), D. Thissen et al. (1988) and R. L. Linn and D. Harnisch (1981). Rudner suggested a measure of the area lying between the two item characteristic…

In order to determine the effectiveness of multidimensional scaling (MDS) in recovering the dimensionality of a set of dichotomously-scored items, data were simulated in one, two, and three dimensions for a variety of correlations with the underlying latent trait. Similarity matrices were constructed from these data using three margin-sensitive…

Attitude measures frequently produce distributions of item scores that attenuate interitem correlations and thus also distort findings regarding the factor structure underlying the items. An actual data set involving 260 adult subjects' responses to 55 items on the Love Relationships Scale is employed to illustrate empirical methods for…

This study used generalizability theory and other statistical concepts to assess the application of the Angoff method to setting cutoff scores on two professional certification tests. A panel of ten judges gave pre- and post-feedback Angoff probability ratings of items of two forms of a professional certification test, and another panel of nine…

This guide is designed to assist those readers of "The Condition of Education" and similar reports who may lack experience or confidence in reading and understanding statistical information. It serves four purposes (1) identifies and describes the principal features of statistical tables and charts; (2) presents a few illustrations of…

This report describes and evaluates the major computer software packages capable of computing standard errors for statistics estimated from complex samples. It first describes the problem and the proposed solutions. The two major programs presently available, SUPER CARP and OSIRIS, are described in general terms. The kinds of statistics available…

In a question and answer format, this guide for parents discusses important issues about standardized reading tests, including the following: (1) scoring systems are obscure; (2) grade level equivalency scores are educationally inaccurate and misleading to child and parent; (3) standardized reading tests are meant to compare groups and not to…

This paper deals with test fairness regarding a test consisting of two parts: (1) a "common" section, taken by all students; and (2) a "variable" section, in which some students may answer a different set of questions from other students. For example, a test taken by several thousand students each year contains a common multiple-choice portion and…

Given the wide individual differences among any group of students and since measurements are always accompanied by errors, the question of how tests should be used in assessing the quality of an educational program is considered. The ways in which educators have dealt with this problem are reviewed, from the systematic examinations of the question…