This study examines the proficiency of Chat GPT, an AI language model, in answering questions on the Situational Judgement Test (SJT), a widely used assessment tool for evaluating the fundamental competencies of medical graduates in the UK. A total of 252 SJT questions from the "Oxford Assess and Progress: Situational Judgement" Test…

Language assessment is increasingly computermediated. This development presents opportunities with new task formats and equally a need for renewed scrutiny of established conventions. Recent recommendations to increase integrated skills assessment in lecture comprehension tests is premised on empirical research that demonstrates enhanced construct…

In this paper, we analyse a large, opportunistic dataset of responses (N = 219,826) to online, diagnostic multiple-choice mathematics questions, provided by 6-16-year-old UK school mathematics students (N = 7302). For each response, students were invited to indicate on a 5-point Likert-type scale how confident they were that their response was…

This study explores the relationship between engagement with an online, free-to-use question-generation application (PeerWise) and student achievement. Using PeerWise, students can create and answer multiple-choice questions and can provide feedback to the question authors on question quality. This provides further scope for students to engage in…

For many years, question choice has been used in some UK public examinations, with students free to choose which questions they answer from a selection (within certain parameters). There has been little published research on choice of exam questions in recent years in the UK. In this article we distinguish different scenarios in which choice…

In this paper we report the results of an experiment designed to test the hypothesis that when faced with a question involving the inverse direction of a reversible mathematical process, students solve a multiple-choice version by verifying the answers presented to them by the direct method, not by undertaking the actual inverse calculation.…

Although learning theories suggest that repeat testing can be highly beneficial for students' retention and understanding of material, there is, so far, little guidance on how to implement repeat testing in higher education. This paper introduces one method for implementing a three-stage model of repeat testing via computer-aided formative…

This paper investigates the impact of online multiple choice questions (MCQs) on students' learning in an undergraduate Accounting module at a British university. The impact is considered from three perspectives: an analysis of how students use the MCQs; students' perceptions expressed in a questionnaire survey; and an investigation of the…

One technology for education whose adoption is currently expanding rapidly in UK higher education is that of electronic voting systems (EVS). As with all educational technology, whether learning benefits are achieved depends not on the technology but on whether an improved teaching method is introduced with it. EVS inherently relies on the…

This paper reports undergraduate student feedback contrasting conventional "Long-answer" examinations with automated multiple-choice question (MCQ) assessment. Feedback was gathered after students had undertaken formative MCQ assessments as a revision aid. Feedback was generally supportive of MCQ summative tests, with 74% expressing a…

This study was carried out to examine students' responses to the use of on-line assessments that included feedback. First year BSc students taking a Cell Biology module undertook such an assessment and were then asked to evaluate the test by completing an anonymous questionnaire. Answers were analysed in light of the respondents' ethnicity and…

Extended matching sets questions (EMSQs) are a form of multiple-choice question (MCQ) consisting of a stem (the question or scenario) with an extended number of possible answers. Although there is no consensus on their absolute format, for the purpose of this paper a multiple-choice question with ten or more alternative answers is considered to be…

Customer-Driven Development is a technique from the software development method called extreme Programming (XP) where customers (most importantly including end users of all levels) are closely involved in the software design and redesign process. This method of producing software suitable for customers has been adapted to help in the production of…

This paper reports on a case study that evaluates the validity of assessing students via a computerized peer-marking process, rather than on their production of an essay in a particular subject area. The study assesses the higher-order skills shown by a student in marking and providing consistent feedback on an essay. In order to evaluate the…