Considerable debate continues regarding the efficacy of the website RateMyProfessors.com (RMP). To date, however, virtually no direct, experimental research has been reported which directly bears on questions relating to sampling adequacy or item adequacy in producing what favorable correlations have been reported. The authors compare the data…

The present study investigates the validity of a 4-point rating scale used to measure the level of preschool children's orientation to literacy during shared book reading. Validity was explored by (a) comparing the children's level of literacy orientation as measured with the "Children's Orientation to Book Reading Rating Scale" (COB)…

Multiple-choice (MC) reading comprehension test items comprise three components: text passage, questions about the text, and MC answers. The construct validity of this format has been repeatedly criticized. In three between-subjects experiments, fourth graders (N[subscript 1] = 230, N[subscript 2] = 340, N[subscript 3] = 194) worked on three…

Item response theory (IRT) methods are generally used to create score scales for large-scale tests. Research has shown that IRT scales are stable across groups and over time. Most studies have focused on items that are dichotomously scored. Now Rasch and other IRT models are used to create scales for tests that include polytomously scored items.…

Psychometric practice does not always converge with the advances of psychometric theory. In order to investigate this gap, the authors focus on the 10 most used psychological tests in Europe, as identified by recent surveys. The article analyzes test manuals published in 6 different European countries for these 10 most used tests. A total of 32…

The Test of English as a Foreign Language (TOEFL) is widely used to screen university applicants for whom English is not their native language. Although the cutoff scores vary, in Ontario those with scores much lower than 550 are rarely admitted to any university. Two exceptions are the University of Western Ontario and its affiliate, Brescia…

In September 2005 the National Foundation for Educational Research (NFER) in association with the Department for Education and Skills (DfES), the Sutton Trust and the College Board, began a five-year research study to examine the validity of an aptitude test in higher education admissions. This report describes and explores the relationships…

This report examines how selection fairness is influenced by the characteristics of a selection instrument in terms of its distribution of item difficulties, level of item discrimination, degree of item bias, and testing strategy. Computer simulation was used in the administration of either a conventional or Bayesian adaptive ability test to a…

This paper considers the question of bias in group administered academic achievement tests, bias which is inherent in the instruments themselves. A body of data on the test of performance of three disadvantaged minority groups--northern, urban black; southern, rural black; and, southwestern, Mexican-Americans--as tryout samples in contrast to…

Two types of computer-administered unit quizzes in a systematic pathology course for second-year medical students were compared. Quizzes composed of questions selected on the basis of a student's ability had higher correlations with the final examination than did quizzes composed of questions randomly selected from topic areas. (Author/JAZ)

Current testing standards call for test developers to provide evidence that testing procedures and test scores, and the inferences made based on the test scores, show evidence of validity and are comparable across subpopulations (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on…

This research attempted to interrelate several methods of producing option weights (i.e., Guttman internal and external weights and judges' weights) and examined their effects on reliability and on concurrent, predictive, and face validity. It was concluded that option weighting offered limited, if any, improvement over unit weighting. (Author/CTM)

The difference in gain scores produced by traditional pretests and those produced by retrospective pretests when compared to posttest scores on the Science Teaching Efficacy Belief Instrument for preservice teachers was investigated in this study. Results indicated that gain scores using the traditional pretest produced significant improvement in…

To compare achievement standards of 1977 with those of 1956, three tests were administered in their original form to all third graders in a large school district. Approximately 3500 students in 1956 and 4500 in 1977 were administered the California Achievement Tests, the California Short Form Tests of Mental Maturity, and the Gates Advanced…

Intended as a comprehensive introduction to the construction and use of foreign language tests, this book utilizes modern linguistic knowledge as a base for scientific language testing. Major attention in testing is focused on such integrated language skills as auditory and reading comprehension, speaking, writing, translation, and over-all…