The purpose of this study was to validate inferences about scores of one task designed to measure subject matter knowledge and three tasks designed to measure aspects of pedagogical content knowledge. Evidence for the validity of inferences was based on two expectations. First, if tasks were sensitive to expertise, we would find group differences.…

This report provides an overview of what was known about alternative assessment at the time that the article was written in 1991. Topics include beliefs about assessment reform, overview of alternative assessment including research knowledge, evidence of assessment impact, and critical features of alternative assessment. The author notes that in…

The goal of this report was to test the use of sensor-based skill measures in evaluating performance differences in rifle marksmanship. Ten shots were collected from 30 novices and 9 experts. Three measures for breath control and one for trigger control were used to predict skill classification. The data were fitted with a logistic regression…

In this article we examine empirical evidence on the criterion, predictive, transfer, and fairness aspects of validity of a large-scale language arts performance assessment, referred to as the Performance Assignment (PA). We use multilevel models to avoid biased inferences that might result from the naturally nested data. Specifically, we examine…

This paper presents the interim results of a set of studies undertaken to develop a much needed methodology for establishing content validity in domain-referenced achievement tests. The study results are presented in the context of the larger issue of the improvement of test design. School teachers, administrators and graduate students were…

Three studies evaluating the validity of the descriptors and exemplars of the National Assessment of Educational Progress (NAEP) as characterizations of the actual mathematics performance of students at achievement levels are reported. Serious inconsistencies were found between actual performance and descriptors and exemplars. Recommendations for…

This article describes performance-based assessment as expounded by its proponents, comments on these conceptions, reviews evidence regarding the technical quality of performance-based assessment, and considers its validity under various policy options. (JDD)

Paper introduces a collection of papers on educational assessment, noting controversies surrounding both old and new assessment methods, discussing the persistent set of technical issues that must be addressed by the research community in measurement, and examining work by the National Center for Research on Evaluation, Standards, and Student…

This booklet is intended to help school personnel, parents, students, and members of the community understand concepts and research relating to achievement testing in public schools. The paper's sections include: (1) test use with direct effects on students (test of certification, selection, and placement); (2) test use with indirect effects on…

A series of studies by the Technical Review Panel (TRP) of the National Assessment of Educational Progress (NAEP) to improve the quality and utility of the NAEP is introduced, with a description of the NAEP, the creation of the TRP, the education policy issues that influenced the selection of research topics, and the relationship among the…

This booklet is planned to help teachers, parents, students, and community members to understand concepts and research relating to testing in public schools. Three general uses--public access, accountability, and standards--relate to testing in its broadest conception. The major purposes of tests are discussed, as are some different types of…

During the past 6 years, under a contract from the National Center for Education Statistics, a Technical Review Panel has overseen and conducted a series of research studies addressing a range of validity questions relevant to the various uses and interpretations of the National Assessment of Educational Progress (NAEP). Study topics included: (1)…

Teachers must learn to distinguish among performance assessments of different quality and appropriateness. Design criteria (cognitive complexity, linguistic appropriateness, content quality and coverage, and meaningfulness) are judged by examining assessment tasks and scoring rubrics. Effects criteria (transfer, generalizability, instructional…

Over the last 20 years, writing about technology (and more recently about its use in assessment systems) has been a joyful experience, in part because the author was usually right about how fast and how unexpected technology progress would be made. In this article, the author suggests to rethink assessment systems from design perspectives and the…