This qualitative study explores the lessons learned from experts who use data visualization in evaluation. Findings suggest that collaborative communication with stakeholders--such as identifying needs, managing expectations, and determining the story to tell--is important before initiating the visualization process. Additionally, evaluators are…

Evaluators must attend to the destructive forces of racialization and racism to contribute to social transformation. Thus, evaluators are called to center culture, context, equity, and social justice during each step of the evaluation process. Here, we focus on the step(s) in which evaluators define program quality and specify evaluative lines of…

The most well-known and controversial version of accountability policies is high-stakes testing. However, various countries employ diverse instruments of 'soft accountability' or low-stakes accountability tools. This paper highlights the main results of a study conducted in Chile that examines the enactment of one of these soft accountability…

Task-based language assessment represents a major component of task-based language teaching syllabi. Current perspectives emphasise the importance of tasks in the assessment process, suggesting that adherence to influential models of language production during task design yields predictable test outcomes. The current study contends that the…

In place of unidirectional transmission of information, discourse on feedback advocates approaches that are more dialogic and promote student agency. The design review is a signature feedback method in architectural education. Although the review's format involves verbal interaction between students and reviewers regarding the students' design…

Background/purpose: Effective supervision is critical for credible educational quality assurance, yet a validated framework for External Quality Assessment Supervisory Units (EQASUs) remains absent in Thailand. This study aimed to develop and validate a competency framework of components and quality indicators to address this gap and enhance EQASU…

Evaluators often lament that the general public does not understand what we do. Yet, there is limited empirical research on what the general public does know--and think--about program evaluation. This article seeks to expand our understanding in this domain by capturing views about evaluation from a demographically representative sample of the U.S…

Few studies have engaged in data-driven investigations of the presence, or frequency, of what could be considered retaliatory assessor behaviour in Multi-source Feedback (MSF) systems. In this study, authors explored how assessors scored others if, before assessing others, they received their own assessment score. The authors examined assessments…

Peer review is integral to the evaluation of grant proposals. Reviewer perceptions and characteristics have received limited study, especially their associations with reviewers' evaluations. This mixed methods study analyzed the unstructured comments of 270 experienced peer reviewers after they scored proposals based on mock overall evaluations…

When forensic interviewers reject children's "Don't know" responses, either by repeating questions or pressuring the children to provide different responses, children may change their subsequent responses. The primary objective of the current study was to examine interviewer reactions following preschool-aged alleged abuse victims'…

This scope-review presents the milestones of how Hierarchical Rater Models (HRMs) become operable to used in automated essay scoring (AES) to improve instructional evaluation. Although essay evaluations--a useful instrument for evaluating higher-order cognitive abilities--have always depended on human raters, concerns regarding rater bias,…

The rise of the digital era has led to a decline in handwriting as the primary mode of communication, resulting in negative effects on handwriting literacy, particularly in complex writing systems such as Chinese. The marginalization of handwriting has contributed to the deterioration of penmanship, defined as the ability to write aesthetically…

Assessment in higher education is often a balancing act between fairness and the inherent complexity of standards, with evaluative judgement playing a crucial role in determining the quality of student work. This study employs a sociomaterial perspective to investigate the often-obscured justifications used by examiners during real-time moderation…

Power has always been an element of the program evaluation field, however, it has traditionally been attributed a negative connotation. With that in mind, power is seen as a force possessed to some extent by different stakeholders that can be imposed over others, and by which certain views, ideas, and decisions can be adopted. Given that program…

The effects of race can manifest in various ways in evaluation contexts, making it critical for evaluators to unpack how race and racism are "complex and destructive forces" for racially minoritized and Indigenous communities. The clarion calls by evaluators on the need for greater attention to issues of race and racism in evaluation…