This article provides a validation framework for research on the development and use of science Learning Progressions (LPs). The framework describes how evidence from various sources can be used to establish an interpretive argument and a validity argument at five stages of LP research--development, scoring, generalisation, extrapolation, and use.…

This article provides a validation framework for research on the development and use of science Learning Progressions (LPs). The framework describes how evidence from various sources can be used to establish an interpretive argument and a validity argument at five stages of LP research--development, scoring, generalisation, extrapolation, and use.…

Despite the call for an argument-based approach to validity over 25 years ago, few examples exist in the published literature. One possible explanation for this outcome is that the complexity of the argument-based approach makes implementation difficult. To counter this claim, we propose that the Assessment Triangle can serve as the overarching…

Kane's paper "Validating the Interpretations and Uses of Test Scores" is the most complete and clearest discussion yet available of the argument-based approach to validation. At its most basic level, validation as formulated by Kane is fundamentally a simply-stated two-step enterprise: (1) specify the claims inherent in a particular interpretation…

According to Kane (2006), the argument-based framework is quite simple and involves two steps. First, specify the proposed interpretations and uses of the scores in some detail. Second, evaluate the overall plausibility of the proposed interpretations and uses. Based on experience gained in developing that validity argument, Chapelle, Enright, and…

The fuzziness of assessing second language speaking performance raises two difficulties in scoring speaking performance: "indistinction between adjacent levels" and "overlap between scales". To address these two problems, this article proposes a new approach, "confidence scoring", to deal with such fuzziness, leading to "confidence" scores between…

The feasibility and efficacy of using self-report response methods with an In-Basket exercise were evaluated in two studies involving 258 managers and 55 college students, respectively. Results suggest that high face validity of the In-Basket exercise can be combined with the scoring ease and objectivity of self-reports. (SLD)

This book is designed to provide fundamental information concerning the procedures necessary to administer, score, interpret, and report findings from the Minnesota Multiphasic Personality Inventory-Adolescent[TM] (MMPI-A), the most widely used objective personality assessment instrument for adolescents. The chapters are: (1) "Overview";…

A recently developed model of validation (T. J. Crooks, M. T. Kane, and A. S. Cohen, 1996) is briefly outlined. It conceptualizes assessment as divided into a chain of eight linked stages: (1) administration; (2) scoring; (3) aggregation; (4) generalization; (5) extrapolation; (6) evaluation; (7) decision; and (8) impact. The model is then used to…

With regard to testing, technology has already played significant roles in the form of scoring technology, analytical practices, and development strategies. Two common functions of technology and testing are currently at work. First is the use of technology to meet existing goals more efficiently, and second is the use of technology to expand the…

The "Standards for Educational and Psychological Testing" (1985) and the "Uniform Guidelines on Employee Selection Procedures" (1978) provide no clear guidance on choosing methods for setting standards on teacher certification tests, although they require that practices be well-documented and produce reasonable results, and…

This technical manual provides all major existing information about the technical characteristics of the Kentucky Instructional Results Information System (KIRIS). The assessment component of the Kentucky Education Reform Act Accountability System was in use between 1991-92 and 1993-94 and was referred to as Cycle One. Two sets of test…

This technical manual provides information about the technical characteristics of the Kentucky Instructional Results Information System (KIRIS), the assessment component of the Kentucky Education Reform Act Accountability System. The time period covered is Accountability Cycle 2, which spanned the school years 1992-93 through 1995-96. Sections of…

Analyses of 10 raters' reasoning during think-aloud interviews provided evidence to support a model of the fundamental processes involved in rating standards-based, nonprescriptive portfolios. This process model provides a framework within which to conceptualize sound-rater reasoning and to identify reasoning that distorts the meaning of scores.…

Student assessment, from the classroom to the state and national levels, is changing. The era of norm-referenced, multiple choice, standardized tests may be coming to an end, to be replaced by other forms of assessment, not all equally good. This guide outlines a vision of what assessment in the service of learning can be and provides ideas,…