In many performance assessments, one or two raters from the complete rater pool scores each performance, resulting in a sparse rating design, where there are limited observations of each rater relative to the complete sample of students. Although sparse rating designs can be constructed to facilitate estimation of student achievement, the…

Many large-scale performance assessments include score resolution procedures for resolving discrepancies in rater judgments. The goal of score resolution is conceptually similar to person fit analyses: To identify students for whom observed scores may not accurately reflect their achievement. Previously, researchers have observed that…

The accreditation process and the federal role in shaping it are clearly of critical importance for ensuring all students have access to quality postsecondary options. And yet, due to the complexity of the relationship among the federal government, accreditors, and institutions, and the opacity of the accreditation process itself, there is little…

Purpose: The purpose of this paper is to compare two years of results for one state's performance-based assessments for principal licensure Performance Assessment for Leaders (PAL). This includes the field trial (2014-2015) and first year of statewide implementation (2015-2016) when passing score requirements and fees were added. Survey results on…

Self-report is currently used as an indicator of professional practice in a variety of fields, including medicine and education. Important to consider, therefore, is the ability of self-report to accurately capture professional practice. This study investigated how well professionals' self-reports of behavior agreed with an expert observer's…

This study was designed to compare teachers certified in South Carolina that were trained in two different methods: through traditional college preparation programs and through the Program of Alternative Certification for Educators (PACE). This study explored three research questions. The first question addressed differences in pedagogical…

This paper examines whether individuals who train themselves to score a performance assessment will rate acceptably when compared to known standards. Research on the efficacy of rater self-training materials developed by the Center for Applied Linguistics for the Texas Oral Proficiency Test (TOPT) is examined. Rater self-materials are described…

Background: Teacher education programs in the State of California are increasingly held accountable for the assessment of teacher candidates' competencies and performance. One salient assessment tool is the utilization of portfolios in teacher education programs (Wolf, 1989; 1991). Among the newest and most innovative templates of portfolios in…

Using videos of physiotherapy students, compared two assessment forms for validity and reliability (the first currently used by an academic program and the second developed from practitioners' perceptions of competence). Also investigated effects of training on assessment decisions. Found wide differences in individual ability to assess students…

This paper presents preliminary, but statistically significant, findings from a study that compares two methods of measuring instructional effectiveness: global evaluation by experts and systematic observation using the SCRIBE Ob.2 software developed at the University of Texas at Austin. Hierarchical instruction of a performance skill…

The effectiveness of alternative procedures to correct for rater leniency/stringency effects was studied when true scores were known. Ordinary least squares, weighted least squares, and imputation of the missing data consistently outperformed averaging the observed ratings; and the imputation technique was superior to the least squares methods.…

Evaluation procedures used in teacher performance assessment systems in 12 states of the Southern Regional Education Board area are described. Twenty-two trained evaluators from 10 states viewed videotaped lessons and applied their own evaluation standards to enable comparison of outcomes of evaluation. Similarities and differences are discussed.…

The New Standards Project is a joint effort of the Learning Research and Development Center (LRDC) at the University of Pittsburgh (Pennsylvania) and the National Center on Education and the Economy toward creation of a national examination system based on performance assessments. This study explored the feasibility of comparing performance on…

Methods that might be used to establish standards of performance that will permit the National Board for Professional Teaching Standards to classify candidate teachers as highly accomplished (worthy of National Board certification) or less than highly accomplished (not worthy of certification) are contrasted. Plans for research on these issues are…

The aesthetic judgments of experts (casting directors and high school drama teachers), theater buffs, and novices were compared as they rated high school students' videotaped performances of Shakespearean monologues. It was hypothesized that theater buffs would represent an intermediate stage on the path to developing expertise in judging acting…