Two aspects of rater accuracy in performance assessment were analyzed: rater stringency/leniency, and rater consistency. Data were obtained from three administrations of an oral certification examination in a health profession. The examination consists of clinical cases in four content areas or subspecialities. A total of 364 candidates were…

Theory and implications for methods of assessing administrative performance in simulated exercises are presented. The rationale is given for the following: (1) developing simulated exercises; (2) measuring behaviors exhibited during the exercises; (3) training evaluators; (4) combining information across exercises; and (5) storing and retrieving…

An instrument for to assessing the extent to which postsecondary vocational education programs are competency-based consisted of 159 items grouped into 8 subscales corresponding to 4 standards: (1) identification and validation of competencies; (2) procedures for assessing student competencies; (3) instructional procedures; and (4) administrative…

This paper makes a distinction between the formative evaluation that can be carried out in the early stages of the development of educational television programs, i.e., from the emergence of the idea through the development of a script, and those that have to be executed in the later stages of development, i.e., during the production process. The…

An internal evaluation was carried out on the governing procedures of the Center for Curriculum and Instruction at the University of Nebraska-Lincoln during the spring semester, 1985-86. The evaluation focused on the major governance committee within the Center (the Center Executive Committee) and its relationship with the two other forms of…

The Center for the Study of Evaluation's (CSE) expository and narrative rating scales have been developed to meet the need for instructionally relevant methods for assessing students' writing competence. Research indicates that large numbers of raters can be trained in the use of these scales and that, during training and independent rating, they…

A case study description of work toward developing an evaluation model for implementation of a statewide principal incentive program illustrates the achieving of consensus when multiple stakeholders' interests were involved. The type of training and background needed by evaluators are also considered. In response to the Educational Improvement Act…

The Louisiana Round Table of Program Evaluators was organized in 1980 among educational evaluators from large parish school systems and the State Department of Education to discuss topics of mutual concern. Since its inception the group has been instrumental in the development and implementation of various workshops directed toward the improvement…

The utilization of program evaluation may be made more effective by means of the application of contemporary persuasion theory. The Elaboration Likelihood Model--a model of cognitive processing, ability, and motivation--was used in this study to test the persuasive effects of source credibility and involvement on message acceptance of evaluation…

Thirty specific standards developed as guiding principles to maintain the utility, feasibility, propriety and accuracy of evaluations in educational programs, projects and materials are presented. Utility Standards include Audience Identification, Evaluator Credibility, Information Scope and Selection, Valuational Interpretation, Report Clarity,…

This paper shares the outcomes of a developmental process designed to create a comprehensive teacher assessment program within a local school district. The paper includes a rationale that helps to explain the need for developing teacher evaluation systems different from most current models, the processes used to create the assessment program, and…

Several issues are related to the use of educational tests. First, test users must be able to choose appropriate tests, interpret scores, and make decisions based on scores. In the field of educational testing, few test users have adequate training in these areas. Second, test makers must clearly specify directions for administration, allowable…

A survey covered categories of budget, organization, staff resources, activities, and reporting techniques and policies; the survey revealed Research and Evaluation (R and E) offices received federal or state funding. Therefore, most retain financial independence from programs evaluated. The best placement of R and E offices in the school…

To examine the impact of program evaluation information upon decision making, case studies were conducted of five different programs: (1) a diagnostic teaching kindergarten program (K-LEAD), funded under the Elementary Secondary Education Act (ESEA) Title I; (2) Project MORE, an ESEA Title IV-C funded junior high school program dealing with school…

Two predictors of coding performance were validated when 26 trained subjects used the Flanders System of Interaction Analysis to code 11 half-hour videotapes of fifth grade classrooms. The Teacher Evaluation Scale, a high-inference rating system based on the Global Rating Scales, was also used. The predictors were scores on a shortened form of the…