Education leaders play a fundamental role in shaping curricular expectations and students' access to books, write education professors Daniel Liou and Kelly Deits Cutler. But it's not easy to know how to respond when leaders are under intense political pressure to pull a book (or topic) from the proverbial shelves. Liou and Deits Cutler share five…

It is necessary to foster teachers' ability to design and implement instructions that encourage students to construct evidence and engage in argumentative discourse. The argument of this paper is that the continuous reflection and discussion as well as regular practice on instructional designs will promote pre-service science teachers' (PSSTs')…

Studies on automatic scoring systems in writing assessments have also evaluated the relationship between human and machine scores for the reliability of automated essay scoring systems. This study investigated the magnitudes of indices for inter-rater agreement and discrepancy, especially regarding human and machine scoring, in writing assessment.…

This article shares the findings of a qualitative case study examining the experiences of teacher leaders as they engaged as teacher evaluators alongside school principals. Data collection included observations and interviews with four teacher leaders and three school principals to answer these research questions: (1) What TDEM structures have…

In recent decades, a number of training environments have moved toward program approaches targeting the development of competencies. Because of their complexity, monitoring the development of those competencies is a considerable challenge. Our hypothesis is that a computerized system could help overcome this challenge if it is well accepted by its…

Instant feedback plays a vital role in promoting academic achievement and student success. In practice, however, delivering timely feedback to students can be challenging for instructors for a variety of reasons (e.g., limited teaching resources). In many cases, feedback arrives too late for learners to act on the advice and reinforce their…

The main objective of this Methods Showcase Article is to show how the technique of adaptive comparative judgment, coupled with a crowdsourcing approach, can offer practical solutions to reliability issues as well as to address the time and cost difficulties associated with a text-based approach to proficiency assessment in L2 research. We…

Expert intuition is increasingly considered to be a valid form of knowledge, and research has proven its effectiveness in judgment and decision making in various fields. Theorists seem to recognize the contributions of intuition within evaluative practice, but it has never been well-documented. This article presents a study on expert intuition,…

Teacher evaluation's relationship with instructional improvement is under-theorized in the literature. To address this gap, this paper uses a conceptual framework rooted in human, social, and material capital to analyze and synthesize findings from research conducted since 2009 on whether and under what conditions teacher evaluation stimulates…

Extending previous research on the Computers Are Social Actors paradigm and the human-to-human interaction script, this study examines the interpersonal impressions of a social robot evaluator versus a human evaluator in a performance evaluation context. A between-subjects experiment was conducted to measure participants' impressions of the…

This paper proposes to use depth perception to represent raters' decision in holistic evaluation of ESL essays, as an alternative medium to conventional form of numerical scores. The researchers verified the new method's accuracy and inter/intra-rater reliability by inviting 24 ESL teachers to perform different representations when rating 60…

Since the George Floyd murder in 2020 and Black Lives Matter campaign, there has been increased interest among evaluators and PIs in Diversity, Equity and Inclusion (DEI) in their projects and evaluations. Our previous research showed a lack of clarity in understanding the distinction between the terms in practice and desire to engage with these…

This article is the fifth in a series on the relationship of evaluation theory and practice. It was commissioned by the Eastern Evaluation Research Society and the American Evaluation Association as part of The Chelimsky Forum, launched in 2013 to honor Eleanor Chelimsky, one of the most influential and respected program evaluators of our era, and…

Professionalising outreach and evaluation work would enhance the quality and rigour of provision, benefit widening participation students and achieve regulatory requirements (Bowes et al. [2019]. "The National Collaborative Outreach Programme End of Phase 1 report for the national formative and impact evaluations." Office for Students;…

According to the standard definition, creative ideas must be both novel and useful. While a handful of recent studies suggest that novelty is more important than usefulness to evaluations of creativity, little is known about the contextual and interpersonal factors that affect how people weigh these two components when making an overall creativity…