The objective structured practical examination (OSPE) is a reliable assessment of practical skills in anatomy teaching. It is often administered as low-stake assessments to track progress at multiple time points in anatomy curricula. Standard-setting OSPEs to derive a pass mark and to ensure assessment quality and rigor is a complex task. This…

Raters must score accurately and consistently for classroom observation scores to be valid. This requires (a) a standard defining when scoring is accurate and consistent enough and (b) measuring and remediating rater performance against that standard. Current practice has focused on this second problem to the exclusion of the first. My goal here…

This article covers the origins, growth, rationale, calibration, and inspiration of an international pool of certified Quality Matters™ (QM) Peer Reviewers. From the beginning in 2003, as a U.S. Department of Education, Fund for the Improvement of Postsecondary Education funded project, QM was developed as a faculty-centered, peer-based approach…

Standard setting methods, like the Bookmark procedure, are used to assist education experts in formulating performance standards. Small group discussion is meant to help these experts in setting more reliable and valid cutoff scores. This study is an analysis of 15 small group discussions during two standards setting trajectories and their effect…

Based on Maryland's 2017-2018 Kindergarten Readiness Assessment (KRA) results, nearly half of all entering kindergarten children show foundational skills indicating they are fully ready for kindergarten, more than a third are approaching readiness, and 18% have emerging readiness skills. Results for the 2017-2018 school year show a slight increase…

This report addresses the comparison of teacher evaluation systems in the context of a particular administrative and legislative challenge: How a state or the federal government could achieve a uniform standard for dispensing funds to school districts for the recognition of exceptional teachers without imposing a uniform evaluation system on those…

Web-based standard setting holds promise for reducing the travel and logistical inconveniences of traditional, face-to-face standard setting meetings. However, because there are few published reports of setting standards via remote meeting technology, little is known about the practical potential of the approach, including technical feasibility of…

The state of Pennsylvania, like many organizations interested in performance improvement, routinely engages in professional development activities. Educators in this hands-on activity engaged in setting meaningful criterion-referenced cut scores for career and technical education assessments using two methods. The main purposes of this study were…

In this article we address the issue of consistency in standard setting in the context of an augmented state testing program. Information gained from the external NRT scores is used to help make an informed decision on the determination of cut scores on the state test. The consistency of cut scores on the CRT across grades is maintained by forcing…

Compared two Angoff standard-setting methods, percentage, and yes-no, in the work of four groups of judges (n=24) given behavioral descriptors or incidents to use in making ratings. Results indicate that passing scores based on percentage estimates were stable from initial to final ratings, but those based on dichotomous (yes-no) ratings had…

Examined intra- and inter-rater consistency of item performance estimated from an Angoff standard setting over 2 years, with 29 panelists one year, and 30 the next. Results provide evidence that item performance estimates were consistent within and across panels within and across years. Factors that might have influenced this high degree of…

The Nielsen/Net report Ratings 2000, reported that in 2002, online usage at work jumped 17 percent year-over-year, driven by female office workers. Nearly 46 million American office workers logged onto the Web, the highest peak since January 2000. It was also predicted that the number of students using the Internet was expected to reach 13.5…

This letter is an interim response to the October 7, 1991 request from the Committee on Education and Labor and the Subcommittee on Elementary, Secondary, and Vocational Education of the House of Representatives asking for a review of the National Assessment Governing Board (NAGB) achievement levels for the National Assessment of Educational…

This document contains papers from a 2-day meeting on identification, evaluation, and recognition of nonformal learning in the European Union. The following papers are included: "Identification, Assessment, and Recognition of Non-Formal Learning: European Tendencies" (Jens Bjornavold); "Why Measure Human Capital?" (Riel…