Introduction: Research in various areas indicates that expert judgment can be highly inconsistent. However, expert judgment is indispensable in many contexts. In medical education, experts often function as examiners in rater-based assessments. Here, disagreement between examiners can have far-reaching consequences. The literature suggests that…

Purpose: SETARA is a well-known university rating tool in Malaysia. The study aims to enhance the transparency, accuracy, and reliability of SETARA assessment instrument by improving its weighting scheme for the domains, sub-domains, criteria, and indicators. Design/methodology/approach: The study utilized a quantitative research design and…

The aim of this study is to show how a many-facet Rasch measurement model (MFRM) can be used for quality control whilst monitoring a musical aptitude examination. The data used in this study was gathered from a musical aptitude examination which was applied in 2019-2020 academic year for selecting teacher candidates to a music education department…

Authentic assessments are seen as a promising response to many of the challenges currently facing Higher Education. Studies have identified shared characteristics of authentic assessments, but it is also argued that the term is vague and subjective. Drawing on existing frameworks we have established a standardised measure to evaluate authenticity…

The field of studies in metacognition points to some limitations in the way the construct has traditionally been measured and shows a near absence of performance-based tests. The Meta-Text is a performance-based test recently created to assess components of cognition regulation: planning, monitoring, and judgment. This study presents the first…

The closed-door PhD and doctoral viva voce--the approach adopted in the United Kingdom--is esteemed by some as being a valuable academic tradition. However, an increasing body of literature and research has raised concerns about the quality, transparency, reliability and validity of this viva format. This systematic literature review aims to…

Background: When learning causal relations, completing causal diagrams enhances students' comprehension judgements to some extent. To potentially boost this effect, advances in natural language processing (NLP) enable real-time formative feedback based on the automated assessment of students' diagrams, which can involve the correctness of both the…

Raters vary from each other in their severity and leniency in rating performance. This study examined the factors affecting rater severity in peer assessments of oral presentations in English as a Foreign Language (EFL), focusing on peer raters' self-construal and presentation abilities. Japanese university students enrolled in EFL classes…

The performance decision tree (PDT; Fulcher et al., 2011) is a rubric style that is applicable to performance assessment, with origins in Upshur and Turner's (1995) empirically derived binary-choice, boundary-definition (EBB) scale. It is easier for raters to assess performance by evaluating multiple binary-choice descriptors. Additionally,…

This study investigates the quality of higher education institutes' (HEIs') administrative services by assessing student satisfaction in the context of Total Quality Management (TQM). Differences between students' perceptions and expectations of administrative service quality are examined and discussed. A questionnaire survey was developed…

The goal of this study is to determine the reliability of the performance points of dentistry students regarding communication skills and to examine the scoring reliability by generalizability theory in balanced random and fixed facet (mixed design) data, considering also the interactions of student, rater and duty. The study group of the research…

Work-integrated learning (WIL) has become an increasingly common feature of higher education curricula. Two aspects of WIL, authenticity and consistency, are valued in different ways by the stakeholders involved. Authenticity, by its very nature, supports the idea of learning being personalized and unique. Consistency, on the other hand, is…

Groupwork is a crucial aspect of work contexts and a key twenty first century skill. Assessment of groupwork provides a persistent challenge for educators in university contexts with students reporting experiences of unfairness from their peers during groupwork. This study developed a novel Peer Assessment Fairness Instrument to explore factors…

Finding valid and reliable ways to assess complex clinical skills within psychology is a challenge. Recently, there have been some examples of applying Objective Structured Clinical Examinations (OSCEs) in psychology for making such assessments. The aim of this study was to examine students' and examiners' perceptions of a digital OSCE in…

Objective Structured Clinical Examinations (OSCEs) have become ubiquitous as a form of assessment in medical education but involve substantial resource demands and considerable local variation. A detailed understanding of the processes by which OSCEs are designed and administered could improve feasibility and sustainability. This exploration of…