Practitioners often struggle to assess reflective learning in the workplace because of difficulties conceptualizing reflection and its effects in the workplace. This article addresses this problem by offering a pragmatic approach to assessment that asks practitioners to specify why they are using reflection, what they are hoping to gain from it,…

In this book the authors explain how to build, and over time improve, the elements of an observation system that equips all observers to identify and develop effective teaching. It is based on the collective knowledge of key partners in the Measures of Effective Teaching (MET) project--which carried out one of the largest-ever studies of classroom…

The implementation of effective instructional materials, such as a core reading program, by a qualified teacher is an important part of improving students' reading achievement. But selecting those instructional materials can be time-consuming. Regional Educational Laboratory (REL) Southeast created this rubric for evaluating reading/language arts…

Human raters are normally involved in L2 performance assessment; as a result, rater behavior has been widely investigated to reduce rater effects on test scores and to provide validity arguments. Yet raters' cognition and use of rubrics in their actual rating have rarely been explored qualitatively in L2 speaking assessments. In this study three…

Purpose: This article provides information about clinical practice guidelines (CPGs) to facilitate their application to the practice of speech-language pathology. CPGs are sets of recommendations based on evidence, including expert clinical opinion, that have been developed by a panel of reviewers. In this article, CPGs are defined and their…

Background: The purpose of this study was to revise and validate the "Individual Child Engagement Record" (ICER), an instrument designed to observe and measure the engagement of children with disabilities in early childhood settings. Method: The ICER was revised with reference to pilot study results and the literature. Inter-observer…

Marking reliability is purported to be produced by having an effective community of practice. No experimental research has been identified which attempts to verify empirically the aspects of a community of practice that have been observed to produce marking reliability. This research outlines what that community of practice might entail and…

The purpose of this study was to determine what trends exist in the identification and description of participants used in data-based studies published in "Adapted Physical Activity Quarterly" and the "Journal of Teaching in Physical Education". Data were analyzed using frequency counts for journals and time periods from the 1980s to 2005 with…

The study, with 79 rehabilitation therapists evaluating 21 single-subject graphs, found that the low interrater agreement often associated with visual analysis of single-subject data may be improved by simple supplements (such as trend lines) to visually inspected charts. (Author/DB)

Case means and case failures in performance-based medical student evaluations were examined to evaluate the consistency of ratings made by two or more standardized patients (SPs) simulating the same case. Results demonstrate a need for caution in interpreting scores obtained from a case checklist completed by multiple SPs. (Author/MSE)

Methods of probability modeling to analyze rater agreement are described, emphasizing their basic similarities and viewing them as variants of a common methodology. Statistical techniques for analyzing agreement data are described to address questions such as how many opinions are required to make a medical diagnosis with necessary accuracy. Kappa…

The use of paraprofessional editors to evaluate student writing, particularly in large college classes, allows teachers to give students more writing practice, provides more individual assistance for students, and helps teachers gain insight into student needs. Adoption of uniform criteria for evaluation also provides consistency and objectivity.…

Interrater reliability in numerical ratings of clerkship performance (n=1,482 students) in five surgery programs was studied. Raters were classified as accurate or moderately or significantly stringent or lenient. Results indicate that increasing the proportion of accurate raters would substantially improve the precision of class rankings. (MSE)

This book offers a descriptive account of the world of the British external examiner: who they are, what they do, how they are appointed, and what problems they face. Findings are based on a questionnaire survey of and personal interviews with examiners. The external examiner is independent of the university awarding the degree and quite…

Thirty-two children aged 7 to 12 participated in a study to determine the reliability of the Ekwall Reading Inventory (ERI) and the Classroom Reading Inventory (CRI). The children were randomly assigned to take one of the two inventories, which were administered by four different specially trained teachers. The study used a test-retest design, in…