Recent developments in the use of large-language models have led to substantial improvements in the accuracy of content-based automated scoring of free-text responses. The reported accuracy levels suggest that automated systems could have widespread applicability in assessment. However, before they are used in operational testing, other aspects of…

Autograding short textual answers has become much more feasible due to the rise of NLP and the increased availability of question-answer pairs brought about by a shift to online education. Autograding performance is still inferior to human grading. The statistical and black-box nature of state-of-the-art machine learning models makes them…

Evaluating written texts is believed to be a time-consuming process that can lack consistency and objectivity. Automated essay scoring (AES) can provide solutions to some of the limitations of human scoring. This research aimed to evaluate the performance of one AES system, Grammarly, in comparison to human raters. Both approaches' performances…

Knowledge tracing (KT) is a method to evaluate a student's knowledge state (KS) based on their historical problem-solving records by predicting the next answer's binary correctness. Although widely applied to closed-ended questions, it lacks a detailed option tracing (OT) method for assessing multiple-choice questions (MCQs). This paper introduces…

This study aimed to reveal the differences in individuals' abilities, their standard errors, and the psychometric properties of the test according to the two methods of applying the test (electronic and paper). The descriptive approach was used to achieve the study's objectives. The study sample consisted of 74 male and female students at the…

Students need more independent roles to evaluate their learning performance, especially in the absence of teachers' feedback due to some constraints such as technology and poor experience. This study investigates the potential of online self-evaluation on students' writing skills in English as a foreign language (EFL). It examines the preparatory…

This study investigated common student errors and underlying difficulties of two groups of Chinese middle school students in an introductory Python programming course using data in the automated assessment tool (AAT) Mulberry. One group of students was from a typical middle school while the other group was from a high-ability middle school. By…

We report on an innovative design of algorithmic analysis that supports automatic online assessment of students' exploration of geometry propositions in a dynamic geometry environment. We hypothesized that difficulties with and misuse of terms or logic in conjectures are rooted in the early exploration stages of inquiry. We developed a generic…

With the iPad-mediated cognitive assessment gaining popularity with school districts and the need for alternative modes for training and instruction during this COVID-19 pandemic, school psychology training programs will need to adapt to effectively train their students to be competent in administering, scoring, an interpreting cognitive…

Using classical test theory and item response theory, this study applied sequential procedures to a real operational item pool in a variable-length computerized adaptive testing (CAT) to detect items whose security may be compromised. Moreover, this study proposed a hybrid threshold approach to improve the detection power of the sequential…

The role of mathematical modelling pertains several disciplines, both STEM and non-STEM, and various fields: education, academy, work, everyday and social life. Despite its importance, it is not uncommon to see university students facing difficulties with the use of Mathematics to create models, even when mathematical entities that play a role in…

In learning, errors are ubiquitous and inevitable. As these errors may signal otherwise latent cognitive processes, tutors--and students alike--can greatly benefit from the information they provide. In this paper, we introduce and evaluate the Systematic Error Tracing (SET) model that identifies the possible causes of systematically observed…

Considerable research effort has been devoted to investigating semantic priming effects, particularly, the locus of those effects. Semantically related primes might activate their target's lexical representation (through automatic spreading activation at short stimulus onset asynchronies (SOAs), or through generation of words expected to follow…

Autograding systems are being increasingly deployed to meet the challenges of teaching programming at scale. Studies show that formative feedback can greatly help novices learn programming. This work extends an autograder, enabling it to provide formative feedback on programming assignment submissions. Our methodology starts with the design of a…

Frequent forced switching between tasks has been shown to reduce switch costs and increase voluntary switch rates. So far, however, the boundary conditions of the influence of forced task switching on voluntary task switching are unknown. Thus, the present study was aimed to test different aspects of generalizability (across items, tasks, and…