Autistic adults who have a history of committing crimes present a major problem for providers of services in terms of legal disposal options and possible interventions, and greater understanding of this group and their associated needs is required. For this reason, we aimed to investigate the face validity of a proposed sub-typology of autistic…

Drawing parallels to classical test theory, this article clarifies the difference between rater accuracy and reliability and demonstrates how category marginal frequencies affect rater agreement and Cohen's kappa. Category assignment paradigms are developed: comparing raters to a standard (index) versus comparing two raters to one another…

Purpose: The differential diagnosis of developmental language disorder (DLD) in bilingual children represents a unique challenge due to their distributed language exposure and knowledge. The current evidence indicates that dual-language testing yields the most accurate classification of DLD among bilinguals, but there are limited personnel and…

This study compared reading rate to reading fluency (as measured by a rating scale). After listening to first graders read short passages, we assigned an overall fluency rating (low, average, or high) to each reading. We then used predictive discriminant analyses to determine which of five measures--accuracy, rate (objective); accuracy, phrasing,…

Rater variation in performance assessment can impact test-takers' scores and compromise assessments' fairness and validity (Crooks, Kane, & Cohen, 1996). Rater variation can also undermine a test's validity and fairness; therefore, it is important to investigate raters' scoring patterns in order to inform rater training. Substantial work has…

This article presents a study conducted by Cornelia Yarbrough and Jennifer Whitaker titled "Analysis of Reviewer Comments About Quantitative Manuscripts Accepted by the "Journal of Research in Music Education"." The study aims to analyze reviewers' comments for quantitative manuscripts with regard to the following categories: section discussed…

The degree of agreement between two raters rating several objects for a single characteristic can be expressed through an association coefficient, such as the Pearson product-moment correlation. How to select an appropriate association coefficient, and the desirable properties and uses of a class of such coefficients--the Euclidean…

A four-level taxonomy is proposed to define the usefulness of rater written comments for supporting letters of recommendation. The taxonomy is used to classify comments on 220 rating forms by 25 raters from two surgery departments regarding performance by third-year medical students. Written comments were classified by the following taxonomy: (1)…

The increasing availability and performance of computer-based testing has prompted more research on the automatic assessment of language and speaking proficiency. In this investigation, we evaluated the feasibility of using an off-the-shelf speech-recognition system for scoring speaking prompts from the LanguEdge field test of 2002. We first…

A study was undertaken to improve the measurement of small team performance within the Army. A provisional taxonomy of team-level performance functions was field-validated; criteria and measures of the functions were developed; and their reliability was examined. The provisional taxonomy, used for observing Army field training exercises, was used…