This introductory article describes how constructed response scoring is carried out, particularly the rater monitoring processes and illustrates three potential designs for conducting rater monitoring in an operational scoring project. The introduction also presents a framework for interpreting research conducted by those who study the constructed…

Translation is one of the items tested in many national English proficiency tests for non-English majors in China because translation competence is regarded as one of the productive language skills which could be used to assess learners' language proficiency. However, the feedback on translation exercises and self-tests are usually provided by…

States and districts can use this tool to create their own plans for continual improvement of an observation system, no matter where they are in their implementation. Included are action steps to improve observation rubrics, observer training, observer assessment, and monitoring. A planning process is described to assess current status, determine…

Nearly all private, government and non-governmental organizations that receive government funding to run social or health promotion programs in the United States are required to conduct program evaluations and to report findings to the funding agency. Reports are usually due at the end of a funding cycle and they may or may not have an influence…

An aspect of assessment which has received little attention compared with perennial concerns, such as standards or reliability, is the role of judgment in marking. This paper explores marking as an act of judgment, paying particular attention to the nature of judgment and the processes involved. It brings together studies which have explored…

E-rater[R] is an automated essay scoring system that uses natural language processing techniques to extract features from essays and to model statistically human holistic ratings. Educational Testing Service has investigated the use of e-rater, in conjunction with human ratings, to score one of the two writing tasks on the TOEFL-iBT[R] writing…

Education Law Section 3012-c requires a new performance evaluation system for classroom teachers ("teachers") and building principals ("principals"). New York State will implement a statewide comprehensive evaluation system for school districts and boards of cooperative educational services (BOCES). The evaluation system is…

Student responses to a large number of constructed response items in three Math and three Reading tests were scored on two occasions using three ways of assigning raters: single reader scoring, a different reader for each response (item-specific), and three readers each scoring a rater item block (RIB) containing approximately one-third of a…

Many approaches to standard-setting use item calibration and student score estimation results to structure panelists' tasks. However, this requires collecting standard-setting judgments after the item analysis results are available. The Scoring Guide Alignment approach collects standard-setting judgments during the scoring sessions from teachers…

The Center for the Study of Evaluation's (CSE) expository and narrative rating scales have been developed to meet the need for instructionally relevant methods for assessing students' writing competence. Research indicates that large numbers of raters can be trained in the use of these scales and that, during training and independent rating, they…

The Department of Measurement, Statistics, and Evaluation (EDMS) at the University of Maryland is working to develop Master's degree programs that are oriented around developing assessment professionals for work in applied settings. Two fundamentally different sets of experiences are being developed: (1) assessment development, administration, and…

Describes the processes in which judges, trying out an integrative approach to portfolio evaluation for the first time, engage as they construct and document their conclusions. The research agenda in which this research is situated is described. (SLD)

The Teacher Evaluation and Assessment Center (TEAC) was established by the Department of Education at the University of South Florida in 1984 to serve the state in the certification of trainers and observers of the Florida Performance Measurement System (FPMS) and to score and report performance evaluations for special programs. This report…

The Assessment Development Laboratory for the National Board for Professional Teaching Standards is developing a certification process with the three components of school site documentation, a content knowledge examination, and assessment center exercises. Activities involved in the development of the exercises and their scoring are described. The…