In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for…

This article sets the stage for the description of a variety of approaches to test reviewing worldwide. It describes the importance of test reviewing as a protection of the public and of society and also the benefits of this activity for test users, who must choose measures to use in particular situations with particular clients at a particular…

The test review process used by the Buros Center for Testing is described as a series of 11 steps: (1) identifying tests to be reviewed, (2) obtaining tests and preparing test descriptions, (3) determining whether tests meet review criteria, (4) identifying appropriate reviewers, (5) selecting reviewers, (6) sending instructions and materials to…

This paper describes the theoretical basis and development of the iSELF: an Internet-tool for Self-Evaluation and Learner Feedback to stimulate self-directed learning in ubiquitous learning environments. In ubiquitous learning, learners follow their own trails of interest, scaffolded by coaches, peers and tools for thinking and learning.…

The purpose of this document is to describe the methods RAND used to analyze achievement for 23 personalized learning (PL) schools for the 2012-13 through 2013-14 academic years. This work was performed at the request of the Bill & Melinda Gates Foundation (BMGF), as part of a multi-year evaluation contract. The 23 schools were selected from a…

Using clickers to gauge student understanding in large classrooms is well documented. Less well known is the effectiveness of using clickers with youth for test taking in large-scale Extension programs. This article describes the benefits and challenges of collecting evaluation data using clickers with a third-grade population participating in a…

Life sciences faculty agree that developing scientific literacy is an integral part of undergraduate education and report that they teach these skills. However, few measures of scientific literacy are available to assess students' proficiency in using scientific literacy skills to solve scenarios in and beyond the undergraduate biology classroom.…

Purpose: The purpose of this paper is to describe how and why Texas A&M University at Qatar (TAMUQ) has developed a system aiming to effectively place students in freshman and developmental English programs. The placement system includes: triangulating data from external test scores, with scores from a panel-marked hand-written essay (HWE),…

This paper describes how a state education system in Australia introduced standards-referenced assessments into its large-scale, high-stakes, curriculum-based examinations in a way that enables comparison of performance across time even though the examinations are different each year. It describes the multi-stage modified Angoff standard-setting…

The state's Race to the Top (RTT) application requires changes to the state's evaluation/accountability system. It requires a teacher effectiveness measure, district effectiveness measure, leader effectiveness measure. Regarding the teacher effectiveness measure, RTT requires linking the data of individual students to individual teachers as part…

This report presents the value-added models that will be used to measure school and teacher effectiveness in the District of Columbia Public Schools (DCPS) in the 2010-2011 school year. It updates the earlier technical report, "Measuring Value Added for IMPACT and TEAM in DC Public Schools." The earlier report described the methods used…

Effective and efficient measurement of the development of skill and knowledge, especially in domains of human activity that involve complex and challenging problems, is important with regard to workplace and academic performance. However, there has been little progress in the area of practical measurement and assessment, due in part to the lack of…

This article presents key steps in the design and analysis of a computer based problem-solving assessment featuring interactive tasks. The purpose of the assessment is to support targeted instruction for students by diagnosing strengths and weaknesses at different stages of problem-solving. The first focus of this article is the task piloting…

This paper presents a comprehensive usability study conducted within the context of a Europe-wide project. The design of the evaluated e-learning platform is based on an innovative approach to the education of young Europeans by integrating into the curricula of a Europe-wide network of 14 schools different state-of-the-art technologies in…

The author describes the testing efforts of Student Services in his school in the light of financial cutbacks, public demand for accountability, student demand for relevance and student apathy toward the counseling center. (Author/HMV)