Background: Meta-analyses of educational interventions have consistently documented the importance of methodological factors related to the choice of outcome measures. In particular, when interventions are evaluated using measures developed by researchers involved with the intervention or its evaluation, the effect sizes tend to be larger than…

By design, randomized experiments (XPs) rule out bias from confounded selection of participants into conditions. Quasi-experiments (QEs) are often considered second-best because they do not share this benefit. However, when results from XPs are used to generalize causal impacts, the benefit from unconfounded selection into conditions may be offset…

In this research investigation of score comparability for language minority students (English language learners [ELLs] and former English language learners), we examined 3 indicators of score comparability (reliability, internal test structure, and differential item functioning) for 4th and 8th grade students who took the NCLB-mandated content…

Extensions of unidimensional appropriateness indices are developed for multiunidimensional tests (multidimensional tests composed of unidimensional subtests). Simulated and real data (scores of 2,978 students on the Armed Services Vocational Aptitude Battery) were used to evaluate the indices' effectiveness in determining individuals who are…

The Kaufman Assessment Battery for Children and Wechsler Preschool and Primary Scale of Intelligence (WPPSI) were administered to Anglo- and Mexican-American children aged four to six. Both scales measured similar, but not identical, intellectual and achievement factors, indicating concurrent validity. Anglo children scored higher on WPPSI verbal…

Meta-analytic research techniques were used to compare the General Aptitude Test Battery (GATB) validities of blacks and non-minorities. The sample consisted of 7,854 black and 15,769 non-minority subjects from 113 Specific Aptitude Test Battery (SATB) validation studies analyzed since 1972. The first approach was to compare average validities…

This report examines how selection fairness is influenced by the characteristics of a selection instrument in terms of its distribution of item difficulties, level of item discrimination, degree of item bias, and testing strategy. Computer simulation was used in the administration of either a conventional or Bayesian adaptive ability test to a…

This paper considers the question of bias in group administered academic achievement tests, bias which is inherent in the instruments themselves. A body of data on the test of performance of three disadvantaged minority groups--northern, urban black; southern, rural black; and, southwestern, Mexican-Americans--as tryout samples in contrast to…

Using both the Behaviorally Anchored Rating Scales (BARS) and the Purdue University Scales, 727 undergraduates rated 32 instructors. The BARS had less halo effect, more leniency error, and lower interrater reliability. Both formats were valid. The two tests did not differ in rate discrimination or susceptibility to rating bias. (Author/GDC)

The results obtained by three test administration modes were examined with a focus upon systematic bias due to lack of standardization of test administration procedures. The My Class Inventory was administered to ninety classrooms of fifth grade students in a large suburban school district. The 45 items on the inventory are distributed over five…

This paper contains a comparison of descriptive information based on analyses of pilot and live administrations of the Alabama High School Graduation Examination (AHSGE). The test is composed of three subject tests: Reading, Mathematics, and Language. The study was intended to validate the test development procedure by comparing difficulty levels…

Two experiments were conducted in Israeli elementary schools to test the assumption that disadvantaged students perform poorly on ability tests because of situational factors related to the testing process. Testing atmosphere and examiner status were examined with respect to verbal and nonverbal ability and intelligence. The assumption was not…

Admission to the California State University system is based on an eligibility index (EI) that considers high school grade point average (GPA) and Scholastic Aptitude Test (SAT) or American College Test (ACT) scores. Whether the SAT adds to the accuracy of prediction of college success or is redundant, and whether it is biased against specific…

The Iowa Tests of Basic Skills were administered to over 600 black and white students in grades six through nine, to determine if the test showed bias against minorities. Outliers were identified from test results. Outliers are items which differ from the central core of test items because they fall outside the range expected from a random…

The criterion-oriented problem of test bias and fairness in selection is compared to the construct-oriented problem of comparability of test scores in cross-cultural research. These problems are shown to have important similarities, and their studies may supplement each other. In a formulation of psychometric criteria for the comparability of test…