Automated Essay Scoring (AES) tools aim to improve the efficiency and consistency of essay scoring by using machine learning algorithms. In the existing research work on this topic, most researchers agree that human-automated score agreement remains the benchmark for assessing the accuracy of machine-generated scores. To measure the performance of…

The COVID-19 pandemic has had a significant impact on high-stakes testing, including the national benchmark tests in South Africa. Current linear testing formats have been criticized for their limitations, leading to a shift towards Computerized Adaptive Testing [CAT]. Assessments with CAT are more precise and take less time. Evaluation of CAT…

The aim of the study was to develop a cost-effective form of online proctoring mechanism that can serve as a scalable and secure solution to cater to high-stake assessments in institutions, with a particular focus on attaining high levels of flexibility and user-friendliness. With reference to existing systems, a self-proctored mechanism is…

[This paper is part of the Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research.] Using a high-stakes thermodynamics exam as the sample (252 students, four multipart problems), we investigate the viability of four workflows for AI-assisted grading of handwritten student solutions. We find that the…

Automated essay scoring (AES) has emerged as a secondary or as a sole marker for many high-stakes educational assessments, in native and non-native testing, owing to remarkable advances in feature engineering using natural language processing, machine learning, and deep-neural algorithms. The purpose of this study is to compare the effectiveness…

This research study focuses on the growing concern of academic misconduct in distance education courses. A quasi-experimental study was conducted to measure the impact of introducing webcam recording software as an online supervision tool for high-stakes exams in two separate online courses (Management and Accounting). Results revealed that…

The ways teachers converse about their work in relation to information and communications technologies (ICTs) are worth studying. We analyse how a teacher converses about her local practices in relation to two spreadsheet algebra programs (SAPs) on variables. During the conversations we noticed that the teacher keeps different policy documents --…

There has been much debate in assessment research about the inclusion of Global English accents in high-stakes listening tests. This study explored test-takers' attitudes toward the inclusion of different English accents in the Duolingo English Test (DET) 2021 test version and their associations with listening test scores. One hundred sixty…

In many high-stakes testing situations, test-takers are not allowed to draw on external writing resources while writing, a practice observed more frequently in classroom settings. This may pose problems with the representativeness of test tasks and score interpretations. This study investigates the domain definition of one particular test known as…

This paper compares and analyzes a selection of popular multilevel tests used for quick accreditation of English as a foreign language worldwide. The paper begins by stating the current need of accreditation of English language competence for both academic and professional matters. It then looks at their defining features and differences. After,…

Nonverbal behavior can impact language proficiency scores in speaking tests, but there is little empirical information of the size or consistency of its effects or whether language proficiency may be a moderating variable. In this study, 100 novice raters watched and scored 30 recordings of test takers taking an international, high stakes…

Cognitive diagnostic models (CDMs) have received much interest within the field of language testing over the last decade due to their great potential to provide diagnostic feedback to all stakeholders and ultimately improve language teaching and learning. A large number of studies have demonstrated the application of CDMs on advanced large-scale…

Individuals are controlled by tests in every advanced society when they want to be admitted in educational courses, to proceed from one stage to the next, or to be given a certificate (Shohamy, 2001b). Accordingly, the present study was carried out to construct and validate educational, social, and psychological consequences questionnaires of…

As a high-stakes standardized test, IELTS is expected to have comparable forms of test papers so that test takers from different test administration on different dates receive comparable test scores. Therefore, this study examined the text difficulty and task characteristics of four parallel academic IELTS reading tests to reveal to what extent…

This paper is concerned with multimodal literacy involving the different kinds of knowledge required to fully access texts with multiple semiotic resources used in reading comprehension test tasks. Such literacy requirements have not drawn researchers' attention to date, mainly because the foreign language teaching and testing project has…