Discusses the need to evaluate the effectiveness of games and simulations that are used for training and educational purposes. Evaluation criteria are described, including validity, reliability, and utility; methods of measuring training device effectiveness are explained; and problems encountered with evaluations are discussed. (30 references)…

A 1990 study of domain-referenced scores from a multiple-station clinical examination for foreign medical graduates investigated identification of essential checklist items, setting of minimum passing scores, consistency of candidate classification, and perceived appropriateness of the number of candidates classified as competent. Results and…

Logical difficulties with existing measures of construct implications are examined, and a new instrument that partially overcomes them--the Logical Relations Grid--is described. Empirical data from a study of 28 children and 47 parents in Australia are given in support of instrument reliability and validity. (SLD)

This rejoinder to Grant Wiggins on performance assessment suggests that true educational reform will undoubtedly be evidenced by something more substantial than pocket folders bulging with student work. Labeling performance tests "authentic" does not ensure their validity, reliability, or incorruptibility. Such tests are neither replacements nor…

The performance of foreign medical school graduates on multistation standardized patient-based tests was used to determine the validity and generalizability of global ratings of their clinical competence made by expert examiners. Results suggest that these ratings can be used as an effective form of assessment in this context. (Author/MSE)

Interrater reliability in numerical ratings of clerkship performance (n=1,482 students) in five surgery programs was studied. Raters were classified as accurate or moderately or significantly stringent or lenient. Results indicate that increasing the proportion of accurate raters would substantially improve the precision of class rankings. (MSE)

The Walker/Severson Systematic Screening for Behavior Disorders measure was revised for use with preschool children. The revision consists of three hierarchical stages of increasingly time-consuming methodologies: (1) teacher rankings, (2) teacher ratings, and (3) direct behavioral observations. Testing with 121 children demonstrated significant…

Evaluation of the Child Dissociative Checklist found it to be a reliable and valid observer report measure of dissociation in children, including sexually abused girls and children with dissociative disorder and with multiple personality disorder. The checklist, which is appended, is intended as a clinical screening instrument and research measure…

Discusses the portability of a Canadian-made educational simulation software package, "The Electronics Workbench," to China that was part of a larger study conducted at the University of Twente (The Netherlands). Evaluation results of the software use in China are presented, including functionality for electronics education, ease of use,…

Portfolio assessment can be a valuable tool in assessing professional proficiency in teachers if appropriate attention is given to issues of reliability and validity. The Teaching Assessment Project at Stanford University has explored portfolios as an alternative to traditional methods of teacher evaluation. (IAH)

A network-flow model is formulated for constructing parallel tests based on classical test theory while using test reliability as the criterion. Practitioners can specify a test-difficulty distribution for values of item difficulties as well as test-composition requirements. An empirical study illustrates the reliability of generated tests. (SLD)

Some of the methods originally used to find relationships between reliability and power associated with a single measurement are extended to difference scores. Results, based on explicit power calculations, show that augmenting the reliability of measurement by reducing error score variance can make significance tests of difference more powerful.…

Two articles discuss the controversy about the relationship between reliability and the power of significance tests in response to the discussion of Donald W. Zimmerman, Richard H. Williams, and Bruno D. Zumbo. Lloyd G. Humphreys emphasizes the differences between what statisticians can do and constraints on researchers. Zimmerman, Williams, and…

Measurement and psychometric quality of the Sternberg task (S. Sternberg, 1966, 1969), a memory search task, was investigated with 78 undergraduates. Individual performance was fairly homogeneous across responses, fairly unstable over time, and fairly stable across stimulus content. Implications for individual differences research are discussed.…

A study tested the validity of the Diagnostic Assessment for the Severely Handicapped-II (DASH-II) for determining the presence of mania (bipolar disorder) in 22 individuals with severe mental retardation. Results found the mania subscale to be internally consistent and able to be used to classify manic and control subjects accurately. (Author/CR)