Quantitative measures of systematic differences in OSCE scoring across examiners (often termed examiner stringency) can threaten the validity of examination outcomes. Such effects are usually conceptualised and operationalised based solely on checklist/domain scores in a station, and global grades are not often used in this type of analysis. In…

Although the principles behind assessment "for" and "as" learning are well-established, there can be a struggle when reforming traditional assessment "of" learning to a program which encompasses assessment "for" and "as" learning. When introducing and reporting reforms, tensions in faculty may…

Assessment of clinical competence is complex and inference based. Trustworthy and defensible assessment processes must have favourable evidence of validity, particularly where decisions are considered high stakes. We aimed to organize, collect and interpret validity evidence for a high stakes simulation based assessment strategy for certifying…

The authors report final-year ward simulation data from the University of Dundee Medical School. Faculty who designed this assessment intend for the final score to represent an individual senior medical student's level of clinical performance. The results are included in each student's portfolio as one source of evidence of the student's…

There is a need for valid methods to assess the readiness for clinical practice of medical graduates. This study evaluates the validity of Utrecht Hamburg Trainee Responsibility for Unfamiliar Situations Test (UHTRUST), an authentic simulation procedure to assess whether medical trainees are ready to be entrusted with unfamiliar clinical tasks…

We developed a Self-Regulation Measure for Computer-based learning (SRMC) tailored toward medical students, by modifying Zimmerman's Self-Regulated Learning Interview Schedule (SRLIS) for K-12 learners. The SRMC's reliability and validity were examined in 2 studies. In Study 1, 109 first-year medical students were asked to complete the SRMC.…

We report a prospective study of clinical observed performance evaluation (COPE) for 197 medical students in the pre-qualification year of clinical education. Psychometric quality was the main endpoint. Students were assessed in groups of 5 in 40-min patient encounters, with each student the focus of evaluation for 8 min. Each student had a series…

One of the impediments to teaching professionalism is unprofessional behavior amongst clinical teachers. No method of reliably assessing the professional behavior of clinical teachers has yet been reported. The aim of this project was to develop and pilot such a tool. Thirty-four desirable professional behaviors in clinical teachers were…

Due to the high-stakes nature of medical exams it is prudent for test agencies to critically evaluate test data and control for potential threats to validity. For the typical multiple station performance assessments used in medicine, it may take time for examinees to become comfortable with the test format and administrative protocol. Since each…

Introduction: Tutorial-based assessment, despite providing a good match with the philosophy adopted by educational programmes that emphasize small group learning, remains one of the greatest challenges for educators working in this context. The current study was performed in an attempt to assess the psychometric characteristics of tutorial-based…

Introduction: This study examined the validity of tensiometry as an objective method of assessing the improvements in the quality of knots and technical performance due to practice on suturing and knot-tying skills. Methods: We evaluated the quality of 1,830 knots produced by undergraduate medical students before training (pre-test), after…

Educators must often decide how many points to use in a rating scale. No studies have compared interrater reliability for different-length scales, and few have evaluated accuracy. This study sought to evaluate the interrater reliability and accuracy of mini-clinical evaluation exercise (mini-CEX) scores, comparing the traditional mini-CEX…

The objective of this paper is to draw attention to four features that distinguish the pedagogy of patient interviewing from the teaching of other clinical skills: (a) students are not naive to the skill to be learned, (b) they encounter role models with a wide variability in interviewing styles, (c) clinical teachers are not usually specialists…

The Tutotest is one of the few structured instruments developed for the assessment of students' learning skills in a problem-based learning (PBL) curriculum. This study was designed to validate the Tutotest in a hybrid PBL curriculum. Forty-four tutors completed 370 evaluations on second to fourth year medical students at the end of the first…

Admissions committees and researchers around the globe have used diligence and imagination to develop and implement various screening measures with the ultimate goal of predicting future clinical and professional performance. What works for predicting future job performance in the human resources world and in most of the academic world may not,…