Describes an approach to addressing issues of validity of inferences and the extrapolation of inferences to target domains beyond the assessment for alternative assessments. Makes the case that in both language testing and educational assessment the roles of language and content knowledge must be considered, and that the design and development of…

The authors describe the significant contributions of Robert Ebel to educational measurement theory and its applications. A biographical sketch details Ebel's roots and professional resume. His influence on classroom assessment views and procedures are explored. Classic publications associated with validity, reliability, and score interpretation…

It is argued that consequential validity is a concept worth considering. The solution to defining "validity" is not to narrow the concept, but to allow for the differential prediction provided by tests in different circumstances. Consequences of the uses and interpretations of test scores are central to their evaluation. (SLD)

The development and validation of professional development resource materials for teachers by the National Council on Measurement in Education and associated professional groups is described. The developed module contains a context-orienting vignette, instruction, practice exercises, hands-on experiences, small group topics, and homework problems.…

Six distinguishable aspects of construct validity are discussed as they apply to performance assessment, emphasizing content, substantive, structural, generalizability, external, and consequential aspects. Taken together, these aspects provide a way to address validity questions in score interpretation and use. (SLD)

The instructional impact of criterion-referenced testing during the past three decades is reviewed, and a method of devising criterion-referenced tests is suggested that departs from the usual model. Adding verification along the lines of panel review that newly constructed test items do elicit appropriate information is suggested. (SLD)

Generalizability of performance-based assessment scores across raters and tasks is examined, focusing on implications of generalizability analyses for specific uses and interpretations of assessment results. Although it seems probable that assessment conditions, task characteristics, and interactions with instructional experiences affect the…

A scenario and accompanying questions and answers are posed to help educators examine possible problems in interpreting a student's test score profile. Profiles developed and used soundly are very helpful, but possible pitfalls in test interpretation must be recognized. (SLD)

An expanded framework for validating tests is needed to include the perspectives of teachers and students as well as of test makers and scientists. The development of educational assessments must take place within an understanding of how tests are used in context. (SLD)

The application of generalizability theory to the reliability and error variance estimation for performance assessment scores is discussed. Decision makers concerned with performance assessment need to realize the restrictions that limit generalizability such as limitations that lead to reductions in the number of tasks possible, rater quality,…

Cohen and Hyman's response contains several misunderstandings of the original article by Mehrens and Kaminski. One frequently wishes to make inferences to a domain from a test, but teaching a specific performance and testing for that performance does not allow for a domain inference. (SLD)

A quiz is presented that may call the attention of school personnel to regression effects in educational measurement, and clarify the meaning of statistical regression for educators in schools. Ten questions and their answers explore regression in the contexts of classroom testing, grading, and performance assessment. (SLD)

Content areas that should be included in testing and measurement courses, subsidiary skills, and means of rationalizing the content and skills are outlined. Areas include basic concepts, assessment uses, assessment planning and development, interpretation of assessments, description of results, evaluation and improvement of assessments, feedback…

The scales of the National Assessment of Educational Progress (NAEP), as constructed, do not yield meaningful criterion-referenced interpretations. Poorly defined NAEP goals and the present knowledge base do not allow the measurement of what examinees can and cannot do. Inappropriate interpretations of NAEP data are discussed, with specific…

The extent to which performance assessments can have unintended and undesirable consequences is not yet clear, but preliminary evidence suggests several issues of bias and fairness. Among these is the sheer difficulty of scoring complex performance assessments, as Vermont's experiences with portfolio assessment illustrate. (SLD)