Despite the positive experiences of many candidates and the existence of many skilled examiners, the closed-door viva has been widely scrutinised on the basis of issues surrounding its reliability and quality. This study aimed to explore the views of examiners regarding this assessment method; specifically, in relation to how the issues with…

What happens when students cannot answer teachers' questions in oral examinations? This study investigates how teachers manage students' insufficient answers in disciplinary oral competence exams (DOCEs) in the secondary school context. Using conversation analysis, we show that teachers either pursue an answer by reformulating it and providing…

The arrival of ChatGPT and other artificial intelligence (AI) has caused considerable concern for educators worldwide, many of whom are still recovering from the impact of the COVID-19 pandemic. Although the debate continues around the use of AI in education, one approach to ensure students are achieving their learning objectives, the oral exam or…

Only a handful of research papers have examined the assessment of student debate activities in higher education, and very little is currently known about how students might perceive a final oral exam in which they have to debate with/against each other. The aim of this study was to examine students' perceptions of participation and learning in…

We propose a novel method for automated short answer grading (ASAG) designed for practical use in real-world settings. The method combines LLM embedding similarity with a nonlinear regression function, enabling accurate prediction from a small number of expert-graded responses. In this use case, a grader manually assesses a few responses, while…

This article focuses on rater severity and consistency and their relation to major changes in the rating system in a high-stakes testing context. The study is based on longitudinal data collected from 2009 to 2019 from the second language (L2) Finnish speaking subtest in the National Certificates of Language Proficiency in Finland. We investigated…

A growing body of literature supports the use of echoic prompts toward conditioning other functional language skills. However, many individuals with autism spectrum disorder do not emit echoic behavior. Identifying the prerequisite skills of an echoic repertoire may be beneficial for intervention planning and clinical decision making. A chart…

While numerical bipolar rating scales may evoke positivity bias, little is known about the corresponding bias in verbal bipolar rating scales. The choice of verbalization of the middle category may lead to response bias, particularly if it is not in line with the scale polarity. Unipolar and bipolar seven-category rating scales in which the…

Prompted by concerns over student use of ChatGPT, staff teaching leadership and teamwork on a large university course experimented with an alternative way of assessing students' learning. Past assessment practices emphasised individual reflection and quality assurance. Aiming for a more relational approach that is aligned with the course's…

Open-text responses provide researchers and educators with rich, nuanced insights that multiple-choice questions cannot capture. When reliably assessed, such responses have the potential to enhance teaching and learning. However, scaling and consistently capturing these nuances remain significant challenges, limiting the widespread use of…

This article explores oral exams in four Norwegian secondary schools. These oral exams are high-stakes assessments, however, knowledge about practice and implications for validity and fairness is scarce. Data for the study consist of video recordings of 36 authentic oral exams in the school subject Norwegian language and literature. Applying…

Many educational institutions transitioned to digital distance-based learning and assessment formats in 2020 due to the severe acute respiratory syndrome--coronavirus 2 (SARS-CoV-2) pandemic. This shift has often been associated with concerns about increased student cheating and heightened stress. In this study, we compared the major course…

Background: Understanding the different factors that determine vocabulary development in young children is essential for the diagnosis and rehabilitation of language disorders in children. Language development is closely related to other cognitive processes such as auditory verbal learning and memory. This research focuses on the development of a…

In this study, we applied different text-mining methods to the originality scoring of the Unusual Uses Test (UUT) and Just Suppose Test (JST) from the Torrance Tests of Creative Thinking (TTCT)--Verbal. Responses from 102 and 123 participants who completed Form A and Form B, respectively, were scored using three different text-mining methods. The…

We revisit the empirically observed positive correlation between DIF and difficulty studied by Freedle and commonly seen in tests of verbal proficiency when comparing populations of different mean latent proficiency levels. It is shown that a positive correlation between DIF and difficulty estimates is actually an expected result (absent any true…