In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for…

Value-added methods refer to efforts to estimate the relative contributions of specific teachers, schools, or programs to student test performance. In recent years, these methods have attracted considerable attention because of their potential applicability for educational accountability, teacher pay-for-performance systems, school and teacher…

Local assessment systems are being marketed as formative, benchmark, predictive, and a host of other terms. Many so-called formative assessments are not at all similar to the types of assessments and strategies studied by Black and Wiliam (1998) but instead are interim assessments. In this article, we clarify the definition and uses of interim…

The Internal Medicine In-Training Examination (ITE) is administered during residency training in the United States as a self-assessment and program assessment tool. Performance on this exam correlates with outcome on the American Board of Internal Medicine Certifying examination. Internal Medicine Program Directors use the United States Medical…

Assessments labeled as formative have been offered as a means to improve student achievement. But labels can be a powerful way to miscommunicate. For an assessment use to be appropriately labeled "formative," both empirical evidence and reasoned arguments must be offered to support the claim that improvements in student achievement can be linked…

In the summer of 2011, the Tennessee Department of Education contracted with the National Institute for Excellence in Teaching (NIET) to provide a four-day training for all evaluators across the state. NIET trained more than 5,000 evaluators intensively in the state model (districts using alternative instruments delivered their own training).…

Background: Validity is a central concern in any assessment, though this has often not been made explicit in the UK assessment context. This article applies current validity theorising, largely derived from American formulations, to national curriculum assessments in England. Purpose: The aim is to consider validity arguments in relation to the…

Background: National curriculum tests have been administered in England for well over a decade. Although reliability evidence has been published, critics have argued that there is not enough evidence (of the right kind) and that test results may be insufficiently reliable. Purpose: This article collates and discusses evidence on the reliability of…

Background: In 2008 primary education in England reached, historically, another important phase in its development. Government reviewed the primary curriculum and some aspects of the national system of assessment. These government reviews coincided with an independent "Primary Review" based at the University of Cambridge to which the…

Purpose: The purpose of this paper is to describe how and why Texas A&M University at Qatar (TAMUQ) has developed a system aiming to effectively place students in freshman and developmental English programs. The placement system includes: triangulating data from external test scores, with scores from a panel-marked hand-written essay (HWE),…

In many school districts, the pressure to raise test scores has created overnight celebrity status for formative assessment. Its powers to raise student achievement have been touted, however, without attending to the research on which these claims were based. Sociocultural learning theory provides theoretical grounding for understanding how…

Author argued that educational evaluation must be broadened considerably from a narrow, quantitative approach to one that also includes qualitative aspects of educational life. (Author/RK)

In this book, educational consultant James Cox offers a smarter way to look at data, putting test scores in context, not in isolation. Rather than focusing on last year's findings for school improvement, educators can learn how to evaluate the processes and practices that lead to those data by using a user-friendly, three-pronged framework for…

These proceedings include four papers on the status of evaluation Michael Scriven ("Evaluation: The State of the Science") examines the general picture of evaluation, identifies a number of trends in the field, and presents several new evaluation concepts. Included are discussions of "responsibility evaluation,""value…