Publication Date
In 2025 | 0 |
Since 2024 | 1 |
Since 2021 (last 5 years) | 1 |
Since 2016 (last 10 years) | 1 |
Since 2006 (last 20 years) | 11 |
Descriptor
Evaluators | 15 |
Interrater Reliability | 15 |
Evaluation Criteria | 5 |
Evaluation Methods | 5 |
Test Reliability | 4 |
School Districts | 3 |
Scoring | 3 |
Student Evaluation | 3 |
Teacher Evaluation | 3 |
Accountability | 2 |
Alignment (Education) | 2 |
More ▼ |
Source
Author
Aaron Zimmerman | 1 |
Abedi, Jamal | 1 |
Beattie, Darrin | 1 |
Bethany L. Miller | 1 |
Brooks, Val | 1 |
Burry, James | 1 |
Century, Jeanne | 1 |
Danielson, Charlotte | 1 |
DiazGranados, Deborah | 1 |
Feldman, Moshe | 1 |
Freeman, Cassie | 1 |
More ▼ |
Publication Type
Reports - Descriptive | 15 |
Journal Articles | 11 |
Guides - Non-Classroom | 1 |
Reports - Evaluative | 1 |
Speeches/Meeting Papers | 1 |
Education Level
Elementary Secondary Education | 4 |
Higher Education | 3 |
Postsecondary Education | 3 |
Adult Education | 1 |
Elementary Education | 1 |
Audience
Practitioners | 1 |
Researchers | 1 |
Teachers | 1 |
Laws, Policies, & Programs
No Child Left Behind Act 2001 | 1 |
Assessments and Surveys
What Works Clearinghouse Rating
Janice Kinghorn; Katherine McGuire; Bethany L. Miller; Aaron Zimmerman – Assessment Update, 2024
In this article, the authors share their reflections on how different experiences and paradigms have broadened their understanding of the work of assessment in higher education. As they collaborated to create a panel for the 2024 International Conference on Assessing Quality in Higher Education, they recognized that they, as assessment…
Descriptors: Higher Education, Assessment Literacy, Evaluation Criteria, Evaluation Methods
Graham, Matthew; Milanowski, Anthony; Miller, Jackson – Online Submission, 2012
As states, districts, and schools transition toward more rigorous educator evaluation systems, they are placing additional weight on judgments about educator practice. Since teacher and principal observation ratings inherently rely on evaluators' professional judgment, there is always a question of how much the ratings depend on the particular…
Descriptors: Interrater Reliability, Evaluators, Observation, Principals
Brooks, Val – Research Papers in Education, 2012
An aspect of assessment which has received little attention compared with perennial concerns, such as standards or reliability, is the role of judgment in marking. This paper explores marking as an act of judgment, paying particular attention to the nature of judgment and the processes involved. It brings together studies which have explored…
Descriptors: Educational Assessment, Test Reliability, Test Validity, Value Judgment
Century, Jeanne; Rudnick, Mollie; Freeman, Cassie – American Journal of Evaluation, 2010
There is a growing recognition of the value of measuring fidelity of implementation (FOI) as a necessary part of evaluating interventions. However, evaluators do not have a shared conceptual understanding of what FOI is and how to measure it. Thus, the creation of FOI measures is typically a secondary focus and based on specific contexts and…
Descriptors: Intervention, Program Implementation, Measurement Techniques, Evaluators
Danielson, Charlotte – Education Digest: Essential Readings Condensed for Quick Review, 2012
The most fundamental reason why teachers are evaluated is because public schools take public money, and the public has a right to expect high-quality teaching. But there are two more basic purposes: (1) to ensure teacher quality; and (2) to promote professional development. The challenge is merging these two purposes of teacher evaluation.…
Descriptors: Teacher Evaluation, Teacher Effectiveness, Faculty Development, Quality Control
Goe, Laura; Holdheide, Lynn; Miller, Tricia – National Comprehensive Center for Teacher Quality, 2011
Across the nation, states and districts are in the process of building better teacher evaluation systems that not only identify highly effective teachers but also systematically provide data and feedback that can be used to improve teacher practice. "A Practical Guide to Designing Comprehensive Teacher Evaluation Systems" is a tool…
Descriptors: Feedback (Response), Teacher Effectiveness, Evaluators, Teacher Evaluation
Feldman, Moshe; Lazzara, Elizabeth H.; Vanderbilt, Allison A.; DiazGranados, Deborah – Journal of Continuing Education in the Health Professions, 2012
Competency-based assessment and an emphasis on obtaining higher-level outcomes that reflect physicians' ability to demonstrate their skills has created a need for more advanced assessment practices. Simulation-based assessments provide medical education planners with tools to better evaluate the 6 Accreditation Council for Graduate Medical…
Descriptors: Performance Based Assessment, Physicians, Accuracy, High Stakes Tests
Beattie, Darrin; Gill, Sue; Wallace, Kelley; Wood, Jim – Practitioner Research in Higher Education, 2009
Using our experience of a work-related learning project, run jointly between Newcastle and Northumbria Universities, in which 600 students annually undertake work placements, this paper focuses on the inherent challenges of using over 30 assessors to assess students. The assessors come from a wide variety of backgrounds and we have had to find…
Descriptors: Student Evaluation, Evaluation Criteria, Interrater Reliability, Educational Theories
Nicastro, Gerilee; Moreton, Kyle M. – Assessment Update, 2008
Western Governors University (WGU) is an online competency-based university in which students demonstrate content competence through a series of assessments. Assessments most often are performance-based or objective assessments that are developed in accordance with specific content objectives. Objective assessments generally assess lower-level…
Descriptors: Evaluators, Performance Based Assessment, Interrater Reliability, Educational Objectives
Sykes, Robert C.; Ito, Kyoko; Wang, Zhen – Educational Measurement: Issues and Practice, 2008
Student responses to a large number of constructed response items in three Math and three Reading tests were scored on two occasions using three ways of assigning raters: single reader scoring, a different reader for each response (item-specific), and three readers each scoring a rater item block (RIB) containing approximately one-third of a…
Descriptors: Test Items, Mathematics Tests, Reading Tests, Scoring

Abedi, Jamal – Multivariate Behavioral Research, 1996
The Interrater/Test Reliability System (ITRS) is described. The ITRS is a comprehensive computer tool used to address questions of interrater reliability that computes several different indices of interrater reliability and the generalizability coefficient over raters and topics. The system is available in IBM compatible or Macintosh format. (SLD)
Descriptors: Computer Software, Computer Software Evaluation, Evaluation Methods, Evaluators

Hurman, John – Language Learning Journal, 1996
Studied the marking characteristics of experienced markers of GCSE role-play to ascertain the extent of variation between the marks they award and to determine whether more intermarker consistency could be obtained with a small increase in time for thought before a particular mark is awarded. Results underlie the importance of reducing the…
Descriptors: Evaluators, Foreign Countries, Interrater Reliability, Oral Language
Oppenheimer, Todd – Education Next, 2007
Educational software makers are often rebuffed by educational authorities, whose endorsements could lead to governmental stamps of approval, and thus explosive sales. But they usually get warmer receptions in the offices of the nation's school superintendents, who are, after all, their primary customers. The system was not supposed to work this…
Descriptors: Federal Legislation, Vendors, Instructional Materials, Computer Software
Quellmalz, Edys S.; Burry, James – 1983
The Center for the Study of Evaluation's (CSE) expository and narrative rating scales have been developed to meet the need for instructionally relevant methods for assessing students' writing competence. Research indicates that large numbers of raters can be trained in the use of these scales and that, during training and independent rating, they…
Descriptors: Evaluation Criteria, Evaluators, Expository Writing, Holistic Evaluation
Novak, Carl D. – 1985
The evaluation team of the Lincoln Public Schools (Nebraska) used the multi-attribution utility technology (MAUT) approach to prioritize potential evaluation projects. The priorities were used to allocate resources to the district's most important projects, and to eliminate or scale down less important projects. The problem was caused initially…
Descriptors: Elementary Secondary Education, Evaluation Criteria, Evaluation Methods, Evaluation Needs