The Performance Assessment for California Teachers (PACT) is a high stakes summative assessment that was designed to measure pre-service teacher readiness. We examined the inter-rater reliability (IRR) of trained PACT evaluators who rated 19 candidates. As measured by Cohen's weighted kappa, the overall IRR estimate was 0.17 (poor strength of…

Classroom observations remain the predominant data source used in teacher evaluations, but little is known about how rater characteristics may affect teachers' scores. For special educators, whose instructional practice requires specialized knowledge and skills, school administrators (i.e., the raters) without experience in special education…

Teacher observations have become a national education phenomenon, mandated by federal policies and promoted by philanthropists. They are crucial components of teacher evaluation systems that often have high stakes for teachers and school systems, but have sparked little innovation. Recent calls to make teacher evaluations better, faster, and…

"The Devereux Student Strengths Assessment Mini" (DESSA-Mini) (LeBuffe, Shapiro, & Naglieri, 2014) efficiently monitors the growth of Social-Emotional Competence (SEC) in the routine implementation of Social Emotional Learning programs. The DESSA-Mini is used to assess approximately half a million children around the world. Since…

In this report we explore the role of reviewer agreement in judgments about alignment between tests and standards. Specifically, we consider approaches to describing alignment that incorporate reviewer agreement information in different ways. The essential questions were whether and how taking into account reviewer agreement changes the picture of…

The appropriateness of the Angoff method (W. H. Angoff, 1971) for setting standards on tests was studied. Evaluators (judges) from California school districts and teacher training institutions reviewed 15 NTE (National Teacher Examinations) Program Specialty Area Tests published by the Educational Testing Service for their appropriateness in…