The award of a PhD degree is based on a rigorous examination process which is unique in that examiners can recommend that the degree is awarded subject to minor or major amendments to the thesis. In their examiner reports, examiners also include formative feedback that is intended to help the doctoral researchers improve their work. Using data…

Rater training is necessary to accurately evaluate fundamental motor skills among children with developmental disabilities (DD). The purpose of this pilot study was to examine the impact of an online rater training program for novice raters on scoring accuracy of the run and two-hand strike skills on the Test of Gross Motor Development-Third…

The ethical guidelines for the American Evaluation Association and the principles of community-based participatory evaluation both state the importance of equitable stakeholder involvement. Regardless of the evaluation approach, however, evaluators are often confronted with gatekeepers, or those who control the access to stakeholders, information,…

Who should evaluate the originality and task-appropriateness of a given idea has been a perennial debate among psychologists of creativity. Here, we argue that the most relevant evaluator of a given idea depends crucially on the level of expertise of the person who generated it. To build this argument, we draw on two complimentary theoretical…

The current paper intends to exploit the Many Facet Rasch Model to investigate and compare the impact of situations (items) and raters on test takers' performance on the Written Discourse Completion Test (WDCT) and Discourse Self-Assessment Tests (DSAT). In this study, the participants were 110 English as a Foreign Language (EFL) students at…

As the development and application of large language models (LLMs) in physics education progress, the well-known AI-based chatbot ChatGPT4 has presented numerous opportunities for educational assessment. Investigating the potential of AI tools in practical educational assessment carries profound significance. This study explored the comparative…

Journals that adopt open peer review (OPR), where review reports of published articles are publicly available, provide an opportunity to study both review content characteristics and quantitative aspects of the overall review process. This study investigates two areas relevant to the quality assessment of manuscript reviews. First, do journal…

Increasingly surveys are using interviewers to collect objective health measures, also known as biomeasures, to replace or supplement traditional self-reported health measures. However, the extent to which interviewers affect the (im)precision of biomeasurements is largely unknown. This article investigates interviewer effects on several…

This paper examines staff's enactment and perceptions of a continuous independent school self-evaluation (SSE) process implemented at a semi-private school network for the past decade. In light of research arguing SSE was perceived and used primarily as a self-inspection or self-regulation tool emphasizing accountability goals; this case suggests…

The profession of evaluation continues to grow, generating more demand for evaluation services than can be fulfilled by the supply of well-trained evaluation practitioners. In this brief forum article, we discuss the current state of evaluator education in the United States and the ways in which university-based programs support the development of…

Evaluating the impact of Academic Literacies teaching (Lea and Street [1998. "Student Writing in Higher Education: An Academic Literacies Approach." "Studies in Higher Education" 23 (2): 157-72. doi:10.1080/03075079812331380364]) is difficult, as it involves gauging whether writers: (1) gain better understanding of what…

Forensic science practitioners compare visual evidence samples (e.g. fingerprints) and decide if they originate from the same person or different people (i.e. fingerprint 'matching'). These tasks are perceptually and cognitively complex--even practising professionals can make errors--and what limited research exists suggests that existing…

Purpose: The present study explored experiences of former markers of undergraduate assignments and examinations at the University of South Africa (Unisa). Methodology: Qualitative method of research was used to gather data. Colaizzi's method (1978) was used to analyze and interpret data. The article's frame of reference was informed by Mezirow's…

Recent research on rater effects in performance assessments has increasingly focused on rater centrality, the tendency to assign scores clustering around the rating scale's middle categories. In the present paper, we adopted Jin and Wang's (2018) extended facets modeling approach and constructed a centrality continuum, ranging from raters…

This article concerns assessors' use of predictions of applicants' ability to learn when assessing admission tests for specialist music teacher education in Sweden. The data consist of stimulated-recall interviews with video-recorded admission tests as stimuli, and the analyses are based on discourse psychology with a focus on the variations,…