Diagnostic classification models (DCMs) classify examinees based on the skills they have mastered given their test performance. This classification enables targeted feedback that can inform remedial instruction. Unfortunately, applications of DCMs have been criticized (e.g., no validity support). Generally, these evaluations have been brief and…

Significant momentum exists for next-generation assessments to increasingly utilize technology to develop and deliver performance-based assessments. Many traditional challenges with this assessment approach still apply, including psychometric concerns related to performance-based tasks (PBTs), which include low reliability, efficiency of…

In this article we examine empirical evidence on the criterion, predictive, transfer, and fairness aspects of validity of a large-scale language arts performance assessment, referred to as the Performance Assignment (PA). We use multilevel models to avoid biased inferences that might result from the naturally nested data. Specifically, we examine…

This collection contains four papers that were presented as part of a short course delivered at the 1991 annual meeting of the Speech Communication Association. The papers discuss various aspects of the development and use of "The Competent Speaker," a public speaking assessment instrument designed for the college sophomore level. The papers and…

A proposed model for the validity of measures of professional competence treats validation as the evaluation of inferences drawn from test scores, focusing on evaluation, generalization, and extrapolation. The model is used to indicate strengths and weaknesses of assessments of professional competence: observations of performance, simulations, and…

A brief history of developments in the assessment of the reliability of credentialing examinations is presented, and some new results are outlined that highlight the interactions among scoring, standard setting, and the reliability and validity of pass-fail decisions. Decision consistency is an important concept in evaluating credentialing…

Explores the relationship between measurement theory and practice, considering five broad categories of: (1) models, assumptions, and terminology; (2) reliability; (3) validity; (4) scaling; and (5) setting performance standards. It must be recognized that measurement is not an end in itself. (SLD)

Results from a performance assessment in which 68 high school students wrote essays support the use of latent variable modeling for estimating reliability, concurrent validity, and generalizability of a scoring rubric. The latent variable modeling approach overcomes the limitations of certain conventional statistical techniques in handling…

An attempt is made is this paper to examine the validity of the American Council on the Teaching of Foreign Languages (ACTFL) scale through a comparison of the scaling of speaking tasks and speech performances by the scale and by a Rasch analysis of judgments made by "naive" persons. The results of the multi-faceted Rasch analysis seem…

The use of portfolio assessment in the Dallas (Texas) Independent School District and the future of portfolio assessment are discussed. A literature review is followed by a description of the development process that preceded the Chapter 1 portfolio assessment of the Dallas schools. Portfolio results are then compared to the standardized measures…

The implementation of competency-based assessment (CBA) under Australia's National Training Framework was evaluated to provide indicators for improvement in CBA policy and practice. The evaluation included three data collection activities: a survey of 258 teachers, trainers, and assessors (response rate, 43%); a survey of 68 Industry Training…