Clinical supervisors are known to assess trainee performance idiosyncratically, causing concern about the validity of their ratings. The literature on this issue relies heavily on retrospective collection of decisions, resulting in the risk of inaccurate information regarding what actually drives raters' perceptions. Capturing in-the-moment…

The present study explores two rating methods for peer assessment (analytical rating using criteria and comparative judgement) in light of concurrent validity, reliability and insufficient diagnosticity (i.e. the degree to which substandard work is recognised by the peer raters). During a second-year undergraduate course, students wrote a one-page…

Education in Doctor of Medicine programs has moved towards an emphasis on clinical competency, with entrustable professional activities providing a framework of learning objectives and outcomes to be assessed within the clinical environment. While the identification and structured definition of objectives and outcomes have evolved, many methods…

Students are often encouraged to learn 'deeply' by abstracting generalizable principles from course content rather than memorizing details. So widespread is this perspective that Likert-style inventories are now routinely administered to students to quantify how much a given course or curriculum evokes deep learning. The predictive validity of…

Practitioners in health sciences education and assessment regularly use a range of psychometric techniques to analyse data, evaluate models, and make crucial progression decisions regarding student learning. However, a recent editorial entitled "Is Psychometrics Science?" highlighted some core epistemological and practical problems in…

In this paper, we will first discuss two current meta-theories dealing with different, aspects of "truth". The first metatheory conceives of truth in terms of coherence (rationality, consistency): a body of knowledge is true when it contains no inconsistencies and has at least some credibility. The second metatheory conceives of truth as…

Standardized narratives or "profiles" can facilitate identification of poor professional behaviour of medical students. If unprofessional behaviour is identified, educators can help the student to improve their professional performance. In an earlier study, based on opinions of frontline teachers from one institution, the authors…

A consistent mentoring approach is key to unlocking the full benefits of mentoring, ensuring effective oversight of mentoring relationships and preventing abuse of mentoring. Yet consistency in mentoring between senior clinicians and medical students (novice mentoring) which dominate mentoring processes in medical schools is difficult to achieve…

Formula scoring (FS) is the use of a don't know option (DKO) with subtraction of points for wrong answers. Its effect on construct validity and reliability of progress test scores, is subject of discussion. Choosing a DKO may not only be affected by knowledge level, but also by risk taking tendency, and may thus introduce construct-irrelevant…

Physicians function as clinicians, teachers, and role models within the clinical environment. Negative learning environments have been shown to be due to many factors, including the presence of unprofessional behaviors among clinical teachers. Reliable and valid assessments of clinical teacher performance, including professional behaviors, may…

Humanistic doctor-patient interaction has been measured for eight years using the Global Patient Assessment (GPA) tool in the national osteopathic clinical skills medical licensure examination. Standardized patients (SPs) apply the GPA tool to rate examinees' competence on doctor-patient communication, interpersonal skills, and professionalism.…

The multiple mini-interview (MMI) has become an increasingly popular admissions method for selecting prospective students into professional programs (e.g., medical school). The MMI uses a series of short, labour intensive simulation stations and scenario interviews to more effectively assess applicants' non-cognitive qualities such as…

In recent years, postgraduate assessment programmes around the world have embraced workplace-based assessment (WBA) and its related tools. Despite their widespread use, results of studies on the validity and reliability of these tools have been variable. Although in many countries decisions about residents' continuation of training and…

We reviewed the literature on instruments for work-based assessment in single clinical encounters, such as the mini-clinical evaluation exercise (mini-CEX), and examined differences between these instruments in characteristics and feasibility, reliability, validity and educational effect. A PubMed search of the literature published before 8…

An essential goal of evaluation is to foster learning. Across the medical education spectrum, evaluation of clinical performance is dominated by subjective feedback to learners based on observation by expert supervisors. Research in non-medical settings has suggested that participants' perceptions of evaluation processes exert considerable…