Despite the widespread use of school-based universal screening systems for social, emotional, and behavioral risk, limited research has examined discrepancies in ratings provided by teachers and their secondary students. Using the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS; teacher report) and mySAEBRS (student report) scores…

The pressing need for English oral communication skills in multifarious contexts today is compelling impetus behind the large number of studies done on oral proficiency interviewing. Moreover, given the recently articulated concerns with the fairness and social dimension of such interviews, parallel concerns have been raised as to how most fairly…

Standard setting (the process of establishing minimum passing scores on high-stakes exams) is a highly evaluative and policy-driven process. It is a common belief that standard setting panels should be diverse and representative. There is concern, however, that panelists with varying characteristics may differentially influence the results of the…

In high-stakes writing assessments, rater training in the use of a rating scale does not eliminate variability in grade attribution. This realisation has been accompanied by research that explores possible sources of rater variability, such as rater background or rating scale type. However, there has been little consideration thus far of…

The practice of handing out excellent grades to students who don't deserve them (grade inflation) is not a new phenomenon. Indeed grade inflation is among the oldest and most difficult issues to address in higher education. The author first studied the impact of grade inflation on student performance on standardized tests at the high school level…

Two factors which may affect the ratings assigned to an essay test are investigated: (1) context effects; and (2) score level effects. Context effects exist in essay scoring if an essay is rated higher when preceded by poor quality essays than when preceded by high quality essays. A score level effect is defined as a change in the score (value)…

The Child Behavior Checklist and related forms were completed for 171 boys referred for school-based assessment resulting from academic and/or behavioral problems. Adolescents consistently underreported behavioral problems relative to parents and teachers regardless of subsequent diagnosis. Implications of these discrepancies in school-based…

The purpose of this paper is to discuss ways in which developmental psychology suffers from the lack of an appropriate technology of measurement and statistical analysis. The paper begins by noting that developmental psychology is the study of change; that individuals develop through a succession of "stages" which are separated by…

This study is an attempt at a cohesive characterization of the concept of essay reliability. As such, it takes as a basic premise that previous and current practices in reporting reliability estimates for essay tests have certain shortcomings. The study provides an analysis of these shortcomings--partly to encourage a fuller understanding of the…

Although the standard-setting method of W. H. Angoff (1971) has broad-based support in the research literature, inconsistencies in the resulting standards do occur. Sources of these inconsistencies are examined in a study of judges, competencies (items), rounds (replications), and the interactions among them. A modified Angoff approach was used to…

Evaluated interscorer reliability of the Spatial Memory subtest, which appears on the Simultaneous Processing scale of the Kaufman Assessment Battery for Children. Responses from 19 gifted children were scored by two independent examiners. Results showed this subtest may be prone to scoring errors because no permanent record of responses exists.…

The purpose of this study was to investigate differences in Texas Teacher Appraisal System (TTAS) scores when considering the scores of the first four individual domains (Instructional Strategies, Management and Organization, Presentation of Subject Matter, and Learning Environment), the sum of the scores of Domains I through IV, and the overall…

A college faculty member who has graded Advanced Placement exam essays on U.S. government and politics, taken mostly by high school juniors and seniors, suggests that high school teachers and college faculty who assess the essays are not the best qualified persons to do so and that despite efforts to ensure consistency, the resulting scores are…

A faculty consultant to the Educational Testing Service for advanced placement (AP) test reading in U.S. government and politics responds to an article criticizing essay evaluation methods and criteria, finding in it a fundamental misunderstanding of the AP reading process and explaining why the essays are subject to less scrutiny for style,…

Across the field of educational assessment the case has been made for alternatives to the multiple-choice item type. Most of the alternative types of items require a subjective evaluation by a rater. The reliability of this subjective rating is a key component of these types of alternative items. In this paper, measures of reliability are…