STEM fields, such as physics, increasingly rely on complex programs to analyse large datasets, thus teaching students the required programming skills is an important component of all STEM curricula. Since undergraduate students often have no prior coding experience, they are reliant on error messages as the primary diagnostic tool to identify and…

Cognitive diagnosis models (CDMs) have been studied as a means of providing detailed diagnostic information about the skills that have been mastered, and the skills that have not, by examinees. Prior research has examined the use of automated test assembly (ATA) alongside CDMs; however, no previous study has investigated how to perform ATA when a…

Computer-assisted analysis of students' written responses to questions is becoming a possibility due to developments in technology. This could make such constructed response questions more feasible for use in large classrooms where multiple choice assessments are often considered a more practical option. In this study, we use a previously…

Automatic item generation can rapidly generate large volumes of exam items, but this creates challenges for assembly of exams which aim to include syntactically diverse items. First, we demonstrate a diminishing marginal syntactic return for automatic item generation using a saturation detection approach. This analysis can help users of automatic…

Mixed studies reviews include empirical studies with diverse designs (qualitative, quantitative and mixed methods). To make the process of identifying relevant empirical studies for such reviews more efficient, we developed a mixed filter that included different keywords and subject headings for quantitative (e.g., cohort study), qualitative…

Troubleshooting is a particular problem-solving process comprising error detection, fault diagnosis, and system restoration. Since automation of systems has become increasingly complex and ubiquitous, troubleshooting skills are crucial to maintain safety and security in a variety of contexts. The planned study aims at examining troubleshooting…

Human operators often experience large fluctuations in cognitive workload over seconds timescales that can lead to sub-optimal performance, ranging from overload to neglect. Adaptive automation could potentially address this issue, but to do so it needs to be aware of real-time changes in operators' spare cognitive capacity, so it can provide help…

Automated question generation is a task to generate questions from structured or unstructured data. The increasing popularity of online learning in recent years has given momentum to automated question generation in education field for facilitating learning process, learning material retrieval, and computer-based testing. This paper report on the…

This article investigates the feasibility of using automated scoring methods to evaluate the quality of student-written essays. In 2012, Kaggle hosted an Automated Student Assessment Prize contest to find effective solutions to automated testing and grading. This article: a) analyzes the datasets from the contest -- which contained hand-graded…

As the demand for skilled data scientists has grown, university level statistics and data science courses have become more rigorous in training students to understand and utilize the tools that their future careers will likely require. However, the mechanisms to assess students' use of these tools while they are learning to use them are not well…

Access to high-quality education at scale is limited by the difficulty of providing student feedback on open-ended assignments in structured domains like programming, graphics, and short response questions. This problem has proven to be exceptionally difficult: for humans, it requires large amounts of manual work, and for computers, until…

As assessments move towards computerized testing and making continuous testing available the need for rapid assembly of forms is increasing. The objective of this study was to investigate variability in assembled forms through the lens of first- and second-order equity properties of equating, by examining three factors and their interactions. Two…

It is imperative to use new technologies in a supportive manner to meet the learners' and teachers' demanding needs as educational environments change in the digital age. The continuous expansion of online learning and distance education opportunities responds to the demands of learners and teachers while pioneering the use of technology in…

Improving pronunciation in second language (L2) learning requires timely and personalized feedback but educators have limited time to dedicate in the classroom. While there are many educational tools available, educators are still struggling to integrate technology in their practices. To address this challenge, this study explores repurposing a…

Writing achievement is crucial for English as a foreign language (EFL) learners due to its impact on language skills, overall proficiency, academic success, creativity, and critical thinking. However, the teacher feedback may be problematic due to timing, subjectivity, overemphasis, and alignment with learning goals. Automated feedback, on the…