In various fields, including college admission, medical board certifications, and military recruitment, high-stakes decisions are frequently made based on scores obtained from large-scale assessments. These decisions necessitate precise and reliable scores that enable valid inferences to be drawn about test-takers. However, the ability of such…

Mastering SQL programming skills is fundamental in computer science education, and Online Judging Systems (OJS) play a critical role in automatically assessing SQL codes, improving the accuracy and efficiency of evaluations. However, these systems are vulnerable to manipulation by students who can submit "cheating codes" that pass the…

This study investigated the optimal feedback intervals for tasks of varying difficulty levels in online testing and whether task difficulty moderates the effect of feedback intervals on student performance. A pre-experimental study with 36 students was conducted to determine the delayed time for providing feedback based on student behavioral data.…

Multidimensional forced-choice (FC) questionnaires have been consistently found to reduce the effects of socially desirable responding and faking in noncognitive assessments. Although FC has been considered problematic for providing ipsative scores under the classical test theory, item response theory (IRT) models enable the estimation of…

Artificial Intelligence is at the heart of modern society with computers now capable of making process decisions in many spheres of human activity. In education, there has been intensive growth in systems that make formal and informal learning an anytime, anywhere activity for billions of people through online open educational resources and…

Remote proctoring, or monitoring test takers through internet-based, video-recording software, has become critical for maintaining test security on high-stakes assessments. The main role of remote proctors is to make judgments about test takers' behaviors and decide whether these behaviors constitute rule violations. Variability in proctor…

This paper describes methods and findings from student cognitive labs, teacher cognitive labs, and test administration observations as evidence evaluated in a validity argument for a computer-based alternate assessment for students with significant cognitive disabilities. Validity of score interpretations and uses for alternate assessments based…

Computational Thinking (CT) skill was a part of the global framework of reference on Digital Literacy for Indicator 4.4.2, widely developed in mathematics and science learning. This study aimed to promote an assessment tool using a two-tier Computerized Adaptive Test (CAT). The study used the Design and Development Research (DDR) method with four…

Curriculum-based measurement of reading (CBM-R) is a common assessment educators use to monitor student growth in broad reading skills and evaluate the effectiveness of instructional programs. Computer-adaptive tests (CATs), such as Star Reading, have been cited as a viable option to formatively assess reading growth. We used Bayesian…

Despite the increasing interest in using AI tools as assistant agents in instructional settings, the effectiveness of ChatGPT, the generative pretrained AI, for evaluating the accuracy of second language (L2) writing has been largely unexplored in formative assessment. Therefore, the current study aims to examine how ChatGPT, as an evaluator,…

Two independent statistical tests of item compromise are presented, one based on the test takers' responses and the other on their response times (RTs) on the same items. The tests can be used to monitor an item in real time during online continuous testing but are also applicable as part of post hoc forensic analysis. The two test statistics are…

The purpose of the present study was to examine the potential that parents could effectively administer an online dyslexia evaluation tool (ODET) to their children. To this end, four groups consisting of parents and trained staff were compared. Sixty-three children (36 females and 27 males) participated. The children in each group were assessed…

Several forced-choice (FC) computerized adaptive tests (CATs) have emerged in the field of organizational psychology, all of them employing ideal-point items. However, despite most items developed historically follow dominance response models, research on FC CAT using dominance items is limited. Existing research is heavily dominated by…

This article explores student perceptions of writing online examinations for the first time during the COVID-19 pandemic. Prior to the pandemic, examinations at an open and distance learning institution in South Africa were conducted as venue-based examinations. From March 2020, all examinations were moved online. Online examinations were…

This study has two main purposes. Firstly, to compare the different item selection methods and stopping rules used in Computerized Adaptive Testing (CAT) applications with simulative data generated based on the item parameters of the Vocational Maturity Scale. Secondly, to test the validity of CAT application scores. For the first purpose,…