Assessment of practical clinical skills is essential in the health fields. Objective Structured Clinical Exams (OSCEs), where examiners assess students performing clinical procedures on simulated patients (actors), are central to the evaluation of practical skills. However, traditional OSCEs require considerable time-investment to administer, and…

We use a natural experiment to show that the presence of an external examiner has both a direct and an indirect negative effect on the performance of monitored classes in standardized educational tests. The direct effect is the difference in the test performance between classes of the same school with and without external examiners. The indirect…

Some methods of determining grade boundaries within examinations, such as awarding, paired comparisons, and rank ordering, entail expert judgements of script quality. We aimed to identify the features of examinees' scripts that most influence judgements in the three methods. For contrasting examinations in biology and English, a Latin square…

The Task-based Language Teaching (TBLT) method has been the proposed teaching method under the current national English curriculum since 2001. However, few studies have investigated in-service teachers' response to this proposed language teaching method. In this study, thirty public school English teachers were recruited in Beijing across school…

Our aim was to deepen understanding of public examinations, exploring how marking task demands influence examiners' cognition and ultimately their marking accuracy. To do this, we identified features of examinations that trigger or demand the use of cognitive marking strategies entailing "reflective" judgements. Kelly's Repertory Grid…

The judgement processes underpinning examination marking are central to achieving fair assessment but are under-researched. This article draws on existing literature and uses additional analysis of data collected in a previous article "Exploring the nature of examiner thinking during the process of examination marking", to start to piece together…

Over recent years, UK medical schools have moved to more integrated summative examinations. This paper analyses data from the written assessment of undergraduate medical students to investigate two key psychometric aspects of this type of high-stakes assessment. Firstly, the strength of the relationship between examiner predictions of item…

The read-aloud accommodation is one of the most frequently used accommodations. Many educators need training to more confidently select, implement, and evaluate the use of the read-aloud accommodation. Planning by special education leaders can help ensure that test day goes smoothly for students who need the read-aloud accommodation.

This paper first analyzed two studies on rater factors and rating criteria to raise the problem of rater agreement. After that the author reveals the causes of discrepencies in rating administration by discussing rater variability and rater bias. The author argues that rater bias can not be eliminated completely, we can only reduce the error to a…

Ultrasound imaging was used to investigate the articulation of the voiceless velar stop [k] in five speakers with compensatory articulation related to cleft palate. The perceptual evaluation of the acoustic realization and the visual assessment of the tongue movement for the target sound were made by three examiners. The analysis revealed a…

To evaluate the implications of deliberate practice when teaching test administration skills, novice, but trained, graduate student examiners administered intelligence tests to a convenience sample of volunteer school-age examinees assigned to a first test session. A second, different convenience sample of volunteer school-age examinees were…

This study investigated factors that could affect inter-examiner reliability in the pronunciation assessment component of speaking tests. We hypothesized that the rating of pronunciation is susceptible to variation in assessment due to the amount of exposure examiners have to nonnative English accents. An inter-rater variability analysis was…

In case of an emergency, a fast and structured patient management is crucial for patient's outcome. The competencies needed should be acquired and assessed during medical education. The objective structured clinical examination (OSCE) is a valid and reliable assessment format to evaluate practical skills. However, traditional OSCE stations examine…

In popular culture, the inventor is often portrayed as a lone tinkerer who emerges from a workshop with some magnificent new device. In reality, most inventions are the collaborative effort of scientists and engineers--usually researchers affiliated with a college or corporation--that result in the discovery, creation, or improvement of either…

Accurate marking is crucial to the reliability and validity of public examinations, in England and internationally. Factors contributing to accuracy have been conceptualised as affecting either marking task demands or markers' personal expertise. The aim of this empirical study was to develop this conceptualisation through investigating the…