Peer review is integral to the evaluation of grant proposals. Reviewer perceptions and characteristics have received limited study, especially their associations with reviewers' evaluations. This mixed methods study analyzed the unstructured comments of 270 experienced peer reviewers after they scored proposals based on mock overall evaluations…

Amid calls for evaluations to advance equity, there are ongoing debates, varied guidance, and limited empirical research on how evaluators practically attend to equity in their work. This article identifies ethical questions--about the right thing to do when there are multiple options--that arise when evaluators attend to equity and factors that…

Psychological theory suggests that evaluators' individual values and traits play a fundamental role in evaluation practice, though few empirical studies have explored those constructs in evaluators. This paper describes an empirical study on evaluators' individual, work, and political values, as well as their personality traits to predict…

Ethics review processes are frequently perceived as extending from codes and protocols rooted in biomedical disciplines. As a result, many researchers in the humanities and social sciences (HASS) find these processes to be misaligned, if not outrightly obstructive to their research. This leads some scholars to advocate against HASS participation…

This article compares the purposes and content of the four foundational documents of the American Evaluation Association (AEA): the Program Evaluation Standards, the AEA Public Statement on Cultural Competence in Evaluation, the AEA Evaluator Competencies, and the AEA Guiding Principles. This reflection on alignment is an early effort in the third…

This study challenges persistent misrepresentations of evaluation as a value-neutral inquiry process by presenting an empirical study that deepens understanding of evaluators' values and how they "show up" in evaluation practice. Through semistructured interviews and inductive analysis, we examined the values advanced by a sample of…

When is it appropriate, and consistent with ethical standards, for evaluators to modify reports in response to "requests" from clients to do so? Some evaluators may view any pressure to make any changes in their reports as an attack on their independence. While, clearly, it is not acceptable for evaluators to be asked to distort or…

The ethical guidelines for the American Evaluation Association and the principles of community-based participatory evaluation both state the importance of equitable stakeholder involvement. Regardless of the evaluation approach, however, evaluators are often confronted with gatekeepers, or those who control the access to stakeholders, information,…

Institutional ethics review procedures aim -- in principle -- to minimise harm and evaluate risks, providing an important space to consider the safety of participants and researchers. However, literature has questioned the effectiveness of the process, particularly for reviewing 'risky' topics in a risk-averse environment. This article reports the…

The primary purpose of Institutional Review Boards (IRBs) is to protect the rights and welfare of human research participants. Evaluation and measurement of how IRBs satisfy this purpose and other important goals are open questions that demand empirical research. Research on IRBs, and the Human Research Protection Programs (HRPPs) of which they…

In this article, I report on examiners' views and experiences of using Pairwise Comparative Judgement (PCJ) and Rank Ordering (RO) as alternatives to traditional analytical marking for GCSE English Language essays. Fifteen GCSE English Language examiners took part in the study. After each had judged 100 pairs of essays using PCJ and eight packs of…

Recurring forms of abuse like domestic violence are unfortunately common. When an individual makes an allegation about their experience, however, there is rarely additional evidence to corroborate their claim. The veracity of the allegation is thus likely to be a central concern in subsequent proceedings. This experiment explored evaluator's…

In this study we show that the perception of lateral variants by Puerto Rican listeners changes according to who the listener believes is speaking. Puerto Rican listeners heard sentences with target words featuring either rhotic [voiced alveolar tap or flap] or lateral [l] (amo[voiced alveolar tap or flap] -- amo[l]) codas, a sociophonetic…

With the rapid development of generative artificial intelligence (AI) frameworks (e.g., the generative pre-trained transformer [GPT]), a growing number of researchers have started to explore its potential as an automated essay scoring (AES) system. While previous studies have investigated the alignment between human ratings and GPT ratings, few…

Instant feedback plays a vital role in promoting academic achievement and student success. In practice, however, delivering timely feedback to students can be challenging for instructors for a variety of reasons (e.g., limited teaching resources). In many cases, feedback arrives too late for learners to act on the advice and reinforce their…