Comparative judgment is gaining popularity as an assessment tool, including for high-stakes testing purposes, despite relatively little research on the use of the technique. Advocates claim two main rationales for its use: that comparative judgment is valid because humans are better at comparative than absolute judgment, and because it distils the…

Scoring procedures for the constructed-response (CR) items in large-scale mixed-format educational assessments often involve checks for rater agreement or rater reliability. Although these analyses are important, researchers have documented rater effects that persist despite rater training and that are not always detected in rater agreement and…

For educational achievement tests that employ multiple-choice (MC) items and aim to reliably classify students into performance categories, it is critical to design MC items that are capable of discriminating student performance according to the stated achievement levels. This is accomplished, in part, by clearly understanding how item design…

Some students are less accurately measured by typical reading tests than other students. By asking teachers to identify students whose performance on state reading tests would likely underestimate their reading skills, this study sought to learn about characteristics of less accurately measured students while also evaluating how well teachers can…

An evaluation design is seen to ensure the presence of certain qualities in an evaluation; values and valued qualities permeate the evaluation design. Mechanisms for evaluations (in-process reviews, follow-up studies, etc.) are dealt with and familiarity with financial, managerial, and ethical procedures are stressed in the evaluation design. (CE)

Compares curriculum evaluation with literary criticism, emphasizing ways the literary concepts of metaphor, point of view, plot, and theme may be applied to curriculum evaluation. Suggests that the use of literary concepts and rhetoric may enable evaluators to describe and judge curricula without distortion or oversimplification. (JG)

This handbook is part of a series prepared by the Center for the Study of Evaluation (CSE) for evaluation workshop participants. Activities are suggested for teams of trainees to practice the use of the five stage evaluation model. The five stages are needs assessment, program planning, implementation evaluation, progress and outcome evaluation.…

This paper analyzes educational evaluation and assesses its value to the educational process. The development of evaluation research and practice is outlined and the paper focuses on a comparison of formative and summative evaluation techniques. Both of these types of evaluation are discussed with an emphasis on formative evaluation. The facets of…

Entertains some general ideas about what arts accountability might look like, keeping in mind the obvious though often overlooked premise that testing should always be in the interests of the children tested. States why accountability is good for reasons beyond its perceived political expediency and outlines some of the major features necessary…

The handbook on progress evaluation was prepared for workshop participants. The Center for the Study of Evaluation (CSE) conducts a series of workshops for administrators and other evaluation team personnel. Specific exercises and activities are suggested using the five stage evaluation model. The phases are needs assessment, program planning,…

A new paradigm for evaluating school restructuring that is based on the teacher-as-researcher concept, or action research, is presented in this paper. The first section compares the traditional and new evaluation paradigms. The old model is competitive, one-dimensional, reactive, and based on external change agents; it appeals to fear and views…

Theories or explanations of educational evaluation are discussed and categorized under two broad methodological headings, the objectivist and subjectivist epistemological orientations. They can be seen as potentially complementary empirical approaches that offer evaluators two methodological orientations to assess educational-reform outcomes.…

The synthesis of historical definitions presented in this paper was designed to trace the development of the process of evaluation, showing how modern educational evaluation has evolved and identifyinq some points of confusion. The importance of the integration of evaluation into the overall educational process is emphasized. (Author/RAO)

Summative evaluation is the final stage of the evaluation framework developed by the Center for the Study of Evaluation. Evaluation is a decision-oriented activity. The four-stage evaluation framework consists of needs assessment, program planning, formative and summative evaluation. The handbook presents exercises in summative evaluation. The…

Currently, there has been substantial interest, in Australia and internationally, in policy activities related to outcomes-based educational performance indicators and their link with growing demands for accountability of teachers and schools. In order to achieve a fair comparison between schools, it is commonly agreed that a correction should be…