This paper describes how work by the United States Office of Personnel Management on the generalizability of employment test validities led to the development of a widely applicable meta-analysis method. The method focuses strongly on estimating the true variance of study correlations and effect size. This validity generalization procedure has…

Drawing from work on the meta-analysis of over 500 validation studies of the General Aptitude Test Battery (GATB) by the U.S. Employment Service, the paper presents a methodological message and a substantive message. Gene Glass's methods, as used by Edwin Ghiselli in personnel selection, ignore study artifacts (sampling error, error of…

As data on the reliability and validity of ratings of infant temperament have accumulated, researchers have begun to ask what caregiver ratings really measure. An argument has been made that ratings of social behavior are less a reflection of enduring individual differences than a measure of rater characteristics and error variance. This study…

This was a replicative study that was initiated with a journeyman level certification instrument for an international union, when industry monitors were observed suggesting to examinees to "go with your first response." The question arose whether this was a researched-based practice. If not, wouldn't this practice inject constant error…

The Test of Standard Written English (TSWE) is a 50-item multiple choice instrument designed to assess the ability of college students to use English. In this study, based upon a sample of 45 students, the TSWE was revalidated with writing samples. The coefficient of 0.54 was most impressive given that the TSWE scores were restricted to those…

This paper isolates and examines separately three distinct sources of error in essay scores: lack of agreement between raters; inconsistencies in performance within mode of discourse, and inconsistencies in performance between modes of discourse. Essay prompts in the Iowa Tests of Basic Skills (ITBS) Writing Supplement were designed to assess…

Criticism of standardized tests is extensive, and a number of claims of test misuse have been made. Test misuse is discussed in six common circumstances. (1) Test users accept the title of a test as an accurate and complete description of the variable being measured, not having full knowledge of the test, its items, and the test manual. A…