The semantics, or meaning, of validity is a fluid concept in educational and psychological testing. Contemporary controversies surrounding this concept appear to stem from the proper location of validity. Under one view, validity is a property of score-based inferences and entailed uses of test scores. This view is challenged by the…

Current educational policies rely on educational assessments. However, the technical aspects of assessments are often unknown to policy makers, which is dangerous because sound assessment policy requires knowledge of the strengths and limitations of educational tests. In this article, we discuss the importance of informing policy makers of…

Recently, student outcomes have bubbled to the top of debates about how to evaluate teaching in community and liberal arts colleges, universities, and professional schools, but even more international attention has been riveted on how outcomes are being used to evaluate teachers and administrators K-12 (Harris, 2012; Rowen & Raudenbush, 2016;…

This paper summarizes an approach to helping future educators to engage with key issues related to the application of measurement-related statistics to learning and teaching, especially in the contexts of science, mathematics, technology and engineering (STEM) education. The approach we outline has two major elements. First, students are asked to…

This book meets a demand in the science education community for a comprehensive and introductory measurement book in science education. It describes measurement instruments reported in refereed science education research journals, and introduces the Rasch modeling approach to developing measurement instruments in common science assessment domains,…

Robert Coe has claimed that three broad conceptions of comparability can be identified from the literature: performance, statistical and conventional. Each of these he rejected, in favour of a single, integrated conception which relies upon the notion of a "linking construct" and which he termed "construct comparability".…

The widespread application of personal computers to educational and psychological testing has substantially increased the number of test administration methodologies available to testing programs. Many of these mediums are referred to by their acronyms, such as CAT, CBT, CCT, and LOFT. The similarities between the acronyms and the methods…

Pupil monitoring systems support the teacher in tailoring teaching to the individual level of a student and in comparing the progress and results of teaching with national standards. The systems are based on the availability of an item bank calibrated using item response theory. The assessment of the students' progress and results can be further…

ACT is often asked whether student scores on the ACT[R] test can be used to make "norm-referenced" or "standards-referenced" comparisons. Norm-referenced interpretations compare students to one another, while standards-referenced interpretations measure student performance against predefined content standards. This brief shows…

Psychometricians continue to introduce new approaches to setting cut scores for educational assessments in an attempt to improve on current methods. In this paper we describe the Item-Descriptor (ID) Matching method, a method based on IRT item mapping. In ID Matching, test content area experts match items (i.e., their judgments about the knowledge…

This document describes the breadth of the research that the ETS (Educational Testing Service) Research & Development division is conducting in 2010. This portfolio will be updated in early 2011 to reflect changes to existing projects and new projects that were added after this document was completed. The research described in this portfolio falls…

When Educational Testing Service (ETS) was first approached by college librarians, professors, and administrators to develop an assessment of Information and Communication Technology (ICT) literacy, there was a resounding concern: incoming students could use technology for entertainment purposes, but many did not have the ICT literacy skills to do…

This article narrates and interrogates some of the responses of a group of English teachers, and their lecturer, to an assignment in a post-graduate course entitled Language and assessment. The assignment required the teachers to respond to a text in a mode other than writing, and to design the assessment of this representation. Three of their…

Discusses the intellectual history supporting the use of educational testing and describes four current tensions in testing: (1) learning and change versus measurement models; (2) quality of information and policy use; (3) precision versus utility; and (4) individual attainment versus standardized attainment. (SLD)

This document describes the breadth of the research being conducted in 2008 by the Research and Development Division at Educational Testing Service (ETS). The research described falls into three large categories: (1) Research supported by the ETS research allocation; (2) Research funded by testing programs at ETS; and (3) Research funded by…