When hearing raters subjectively evaluated the signed and spoken output of 25 sign language interpreters, rater agreement was between 0.52-0.86; the correlation between subjective and objective evaluation was between 0.59-0.79. Raters were unsuccessful in identifying which interpreters had deaf parents. (CB)

Questionnaire data from 349 directors of special education and state implementation plans were analyzed to determine which professionals were most often included on multidisciplinary evaluation teams for diagnosing mental retardation. The data revealed variations in type and number of personnel comprising these teams; however, team size…

Ethical issues relating to naturalistic evaluation are addressed, focusing on the role of the evaluator, problems of privacy and data gathering techniques, and issues relating to the use/abuse of findings. Benefits and costs of the naturalistic approach to program evaluation are also identified. (EGS)

The Safe Schools/Healthy Students (SS/HS) Initiative, funded by the U.S. Departments of Education, Health and Human Services, and Justice, is an ambitious federal effort to improve the health and well-being of children and adolescents. The Initiative focuses on integrated prevention and treatment programs, strategies, and services designed to…

This paper describes a study on rater training that involved the analysis of ratings given to English-as-a-Second-Language (ESL) compositions by 8 inexperienced and 8 experienced raters both before and after rater training, using FACETS (Linacre, 1990, 1993), which provides measures of rater severity and consistency. The testing text was a…

Legislation passed during the 2001 Iowa legislative session established the Student Achievement and Teacher Quality Program, Senate File 476. This legislation requires the Iowa Department of Education (DE) to annually report the statewide progress on the following: (1) Student achievement scores in mathematics and reading at the fourth and eighth…

In a simulated adversary-advocacy evaluation of a computer-assisted school guidance program, the evaluation audience's reaction to different forms of data, the order of argument presentations, and the rated attitude of the evaluator are shown to influence audience agreement with evaluation recommendations. (CM)

Examines the effects of organizational differences and rater differences on performance appraisals. Self, peer, and supervisory ratings of performance for nurses and for resident advisors were used. Both organization and rater differences have significant, independent effects on performance ratings and may restrict the generality of findings.…

Several concepts and themes fundamental to program evaluation are identified: value positions, concept of fairness, and the lack of an ideal applicable to all situations. Evaluation references are selected covering the topics: evaluation approaches, evaluation issues, evaluation in higher education, and administrator evaluation. (MLW)

Students, professional educators, and business personnel rated simulated evaluation reports. Evaluator's sex and professional level of the audience influenced agreement with report recommendations; use of report data did not. (CP)

Presented a distinction between concepts of face validity and item subtlety. Trait categories were differentially accessible to individual judges. Higher criterion validities were associated with less subtle and more face-valid items. Results support a rational approach to test construction and emphasize the use of relevant test item content.…

Predicted that the accuracy of evaluating videotape replay of one's own behavior is related to the defense mechanism clusters of the evaluator. Results confirmed the prediction. Furthermore, subjects characterized by projection clusters tended to produce the greatest distortions, and those characterized by turning-against-self clusters evaluated…

Reports on the interrater reliability of a language arts portfolio assessment in the primary grades of the Rochester (New York) school system. Results from approximately 400 primary grade portfolios rated by 2 raters show that teachers can assess their own students' work reliably. (SLD)

This article provides a case study of the use of emergent design, which asks evaluators to adapt their inquiry processes as understanding develops, in the evaluation of a family support program, tracing the emergence of evaluation procedures through three phases. Two phases produced data that policymakers found very helpful. (SLD)

Examinations were analyzed using the Rasch model to determine differences in judge severity and grading period stringency for (1) essay examination (subjects were 12 judges and 32 examinees); (2) clinical examination (subjects were 18 judges and 217 examinees); and (3) oral examination (subjects were 46 judges and 270 examinees). (SLD)