This paper analyzes the nature and perceived effects of mid-stakes testing (known as the EQAO) in Ontario, Canada. Ontario's mid-stakes tests were meant to ensure accountability and transparency, and assure system-wide improvement, while avoiding the negative effects and perverse incentives of their high-stakes counterparts. The paper provides new…

Fairness in assessment is seen as increasingly important but there is a need for greater clarity in use of the term 'fair'. Also, fairness is perceived through a range of 'lenses' reflecting different traditions of thought. The lens used determines how fairness is seen and described. This article distinguishes different uses of 'fair' which have…

This National Academy of Education (NAEd) volume provides guidance to key stakeholders on how to accurately report and interpret comparability assertions concerning large-scale educational assessments as well as how to ensure greater comparability by paying close attention to key aspects of assessment design, content, and procedures. The goal of…

The field of educational measurement has evolved considerably since the first doctoral programs were established. In response, programs have typically tacked on courses that address newly developed theories, methods, tools, and techniques. As our review of current programs evidences, this approach produces artificial distinctions among topics and…

Testing programs are often classified as high or low stakes to indicate how stringently they need to be evaluated. However, in practice, this classification falls short. A high-stakes label is taken to imply that all indicators of measurement quality must meet high standards; whereas a low-stakes label is taken to imply the opposite. This approach…

A common interpretation of achievement test results is that they provide measures of achievement that are much like other measures we commonly use for height, weight, or the cost of goods. In a limited sense, such interpretations are correct, but some nuances of these interpretations have important implications for the use of achievement test…

Many standardized tests are now administered via computer rather than paper-and-pencil format. The computer-based delivery mode brings with it certain advantages. One advantage is the ability to adapt the difficulty level of the test to the ability level of the test taker in what has been termed computerized adaptive testing (CAT). A second…

At the outset of his thoughtful and thought-provoking article, Haertel (this issue) clearly identifies the issue with which he will be dealing: The disjunct, or gap, in current approaches to evaluating the merits of a given test, between the intended uses of that test and the validity of its score-based interpretations. The author thinks that…

E. L. Thorndike contributed significantly to the field of educational and psychological testing as well as more broadly to psychological studies in education. This article follows in his testing legacy. I address the escalating demand, across societal sectors, to measure individual and group competencies. In formulating an approach to measuring…

What is validity? A simple question but apparently one with many answers, as Paul Newton highlights in his review of the history of validity. The current definition of validity, as entertained in the 1999 "Standards for Educational and Psychological Testing" is indeed a consensus, one between the classical notion of attributes, and measures…

This focus article provided the author with an opportunity to unpack the consensus definition of validity and to explore its implications in the light of recent debates. He proposed an elaboration of the consensus definition, which was intended to express the spirit of the "Standards for Educational and Psychological Testing" with increased…

In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for…

The field of educational testing has become increasingly important for providing different stakeholders and decision-makers with information. This paper discusses basic standards for methodological approaches used in measuring literacy skills among adults. The authors address the increasing interest in skills measurement, the discourses on how…

This paper presents a noncompensatory latent trait model, the multicomponent latent trait model for diagnosis (MLTM-D), for cognitive diagnosis. In MLTM-D, a hierarchical relationship between components and attributes is specified to be applicable to permit diagnosis at two levels. MLTM-D is a generalization of the multicomponent latent trait…

The measurement community needs to better understand how to interact with the media to effectively disseminate important findings from educational testing efforts. To this end, the current paper will review media coverage of educational testing and related issues and elaborate on areas of concern and opportunities for improved communication…