Suggests that a modification of traditional linking is necessary when tests consist of constructed response items judged by raters and a possibility of year-to-year variation in rating discrimination and severity exists. Illustrates this situation with an artificial example. (SLD)

Self, subordinate, peer, and supervisor ratings of 1,080 managers in education, military, government, manufacturing, finance, and health were analyzed for leniency, interrater agreement, and effectiveness. In the private sector, more poor performing managers tended to overestimate their performance. Interrater agreement was lowest in government…

Evaluates the effects of rater reliability of common IQ and achievement tests on subsequent learning disorder eligibility determinations, particularly with respect to difficulty level of individual subtests and expertise of the scorer. The study corroborates previous findings of strong interrater reliability on most subtests of common IQ and…

In this study, 30 judges identified disfluency types they perceived in audiovisually recorded speech stimuli, first individually and then with a partner. Although intrapair and interpair agreement was higher in the partner than the individual condition, agreement for occurrences still averaged below 50 percent. Findings suggest caution in use of…

A coefficient of unfairness in the allocation of goods to people can be extended to measuring consensus among judges. The notion of relative variability underlies the formation of these measures.

This article discusses the rationale for, and explicates the process used in, developing differentiated authentic assessments for middle school classrooms (many of which contain gifted students) that are aligned with state academic standards. The assessments were developed based on learner-centered psychological principles and revised based on a…

This article addresses the problem of accounting overall multivariate chance-corrected interobserver agreement when targets have been rated by different sets of judges (not necessarily equal in number). The proposed approach builds on Janson and Olsson's multivariate generalization of Cohen's kappa but incorporates weighting for number of judges…

This paper comments on the response offered by Szechtman and Woody to Taylor et al's initial comments on Szechtman and Woody's original article. Taylor et al highlight one problem with their model that Woody and Szechtman seem to think is unimportant: the treatment relevance of their model. The analogy of aspirin and colds was used, suggesting…

This study describes a rubric supporting fast and reliable assessment of preservice teacher electronic portfolios. The assessment calls for raters to quickly scan a portfolio to gain an overall impression, then dichotomously score a large number of indicators (e.g., educational philosophy, educational technology use, imaginative use of…

A key feature of autism is restricted repetitive behavior (RRB). Despite the significance of RRBs, little is known about their phenomenology, assessment, and treatment. The Repetitive Behavior Scale-Revised (RBS-R) is a recently-developed questionnaire that captures the breadth of RRB in autism. To validate the RBS-R in an independent sample, we…

This article describes the development of the Comprehensive Assessment of Team Member Effectiveness. The authors used the teamwork literature to create potential items, which they tested using two surveys of college students (Ns = 2,777 and 1,157). The authors used exploratory factor analysis and confirmatory factor analysis to help them select…

Nutrition screening and early intervention in home-bound older adults are key to preventing unfavourable health outcomes and functional decline. This pilot study's objectives were (a) to test the reliability of the Elderly Nutrition Screening Tool (ENS [C]) when administered by dietician-trained and supervised nutrition volunteers, and (b) to…

We conducted generalizability studies to examine the extent to which ratings of language arts performance assignments, administered in a large, diverse, urban district to students in second through ninth grades, result in reliable and precise estimates of true student performance. The results highlight three important points when considering the…

This review examines interventions intended to improve the literacy functioning of adolescent students with emotional and/or behavior disorders. Seventeen studies met inclusion criteria and included a variety of interventions designed to affect a variety of literacy areas, including spelling, writing, and reading fluency. Findings from these…

Until recently, research studies have implied that domestic violence does not affect Asian American and immigrant communities, or even Asians abroad, because ethnicity or culture has not been addressed. In this content analysis, the authors examined trends in publications in leading scholarly journals on violence relating to Asian women and…