Textbook calculations of statistical power or sample size follow from formulas that assume that the variables under consideration are measured without error. However, in the real world of behavioral research, errors of measurement cannot be neglected. The determination of sample size is discussed, and an example illustrates blocking strategy.…

To control for response bias in student ratings of college teachers, an index of rater error was used that was theoretically independent of actual performance. Partialing out the effects of this extraneous response bias enhanced validity, but partialing out overall effectiveness resulted in reduced convergent and discriminant validities.…

As part of a longitudinal study, Matching Familiar Figures Test (MFFT) performance was assessed at age 11 and related to California Child Q-Sort evaluations obtained both concurrently and at age 14. Offers evidence for a "competence" interpretation rather than a "conceptual tempo" interpretation of the MFFT and strongly…

Two new indices of stability of content-referenced standard-setting results are presented, relating variability of judges' decisions to the variability of candidate scores and to the reliability of the test. These indices are used to indicate whether scores resulting from a standard-setting study are of sufficient precision. (Author/LMO)

Describes an experiment designed to teach students to apply the same statistical awareness to instrumentation they commonly apply to classical techniques. Uses propagation of error techniques to pinpoint instrumental limitations and breakdowns and to demonstrate capabilities and limitations of volumetric and gravimetric methods. Provides lists of…

A procedure for obtaining maximum likelihood trait estimates from number-correct (NC) scores for the three-parameter logistic model is presented. It produces an NC score to trait estimate conversion table. Analyses in the estimated true score metric confirm the conclusions made in the trait metric. (Author/DWH)

The purpose of this paper is to evaluate the use of MC2G program to teach certain topics in statistics education. MC2G is a program written in Pascal Delphi by Gordon Brooks of Ohio University based on Monte Carlo studies. MC2G provides students opportunity to practice important topics in an introductory statistics course, such as power, Type I…

Different item response theory (IRT) models may be employed for item calibration. Change of testing vendors, for example, may result in the adoption of a different model than that previously used with a testing program. To provide scale continuity and preserve cut score integrity, item parameter estimates from the new model must be linked to the…

This paper presents a number of arguments for the increased importance of within-state district-level data in systematic assessments in the organizational structure of schools as educational institutions. The major question is whether the Schools and Staffing Survey (SASS) should shift its focus toward more macro-institutional district level…

A simulation study of methods of assessing differential item functioning (DIF) in computer-adaptive tests (CATs) was conducted by Zwick, Thayer, and Wingersky (in press, 1993). Results showed that modified versions of the Mantel-Haenszel and standardization methods work well with CAT data. DIF methods were also investigated for nonadaptive…

Study analyzed the results of a college rule regulating the maximum number of units students were allowed to carry and its effect on academic performance. (Author/RK)

A reliability coefficient for criterion-referenced tests is developed from the assumptions of classical test theory. The coefficient is based on deviations of scores from the criterion score, rather than from the mean. (Author/CK)

An alternative interpretation of Livingston's reliability coefficient (see TM 500 487) is based on the notion of the relation of the size of the reliability coefficient to the range of talent. (Author/CK)

The generalizability theory approach to the estimation of reliability is outlined, and a design is developed in which systematic variations in behavior over differing situations are separated from random fluctuation. Three coefficients of reliability are proposed. (CK)