This report describes the process of developing and validating the enhanced ACT. The report describes the changes made to the test content and the processes by which these design decisions were implemented. The authors describe how they shared the overall scope of the enhancements, including the initial blueprints, with external expert panels,…

The highly adaptive testing (HAT) design is introduced as an alternative test design for the Programme for International Student Assessment (PISA). The principle of HAT is to be as adaptive as possible when selecting items while accounting for PISA's nonstatistical constraints and addressing issues concerning PISA such as item position effects.…

Multistage adaptive testing (MST) has been recently adopted for international large-scale assessments such as Programme for International Student Assessment (PISA). MST offers improved measurement efficiency over traditional nonadaptive tests and improved practical convenience over single-item-adaptive computerized adaptive testing (CAT). As a…

Partially automated assessment is implemented via the 'Printable worksheet' mode in the Numbas e-assessment system to create a mathematical modelling worksheet which is individualised with random parameters but completed and marked as if it were a non-automated piece of coursework, preserving validity while reducing the risk of academic misconduct…

The purpose of this study was to compare calibration and linking methods for placing pretest item parameter estimates on the item pool scale in a 1-3 computerized multistage adaptive testing design in terms of item parameter recovery. Two models were used: embedded-section, in which pretest items were administered within a separate module, and…

Online calibration is a key technology for item calibration in computerized adaptive testing (CAT) and has been widely used in various forms of CAT, including unidimensional CAT, multidimensional CAT (MCAT), CAT with polytomously scored items, and cognitive diagnostic CAT. However, as multidimensional and polytomous assessment data become more…

Computer-based testing (CBT) is becoming an increasingly popular format of assessment in educational settings. If students face a digital divide in terms of access to computers at school and at home, CBT may exacerbate measured student achievement gaps. In this paper, we use the rollout of CBT in South Carolina starting in 2015 to investigate its…

To improve the calibration accuracy and calibration efficiency of cognitive diagnostic computerized adaptive testing (CD-CAT) for new items and, ultimately, contribute to the widespread application of CD-CAT in practice, the current article proposed a Gini-based online calibration method that can simultaneously calibrate the Q-matrix and item…

The article deals with the results of mathematics examinations at the University of Finance and Administration in Prague before, during, and immediately after the COVID-19 pandemic-related restrictions. The first objective is to evaluate whether the non-standard forms of testing (correspondence and online), used on an emergency basis during the…

The digital transformation of education is greatly accelerating in various computer-supported applications. As a particularly prominent application of the human-machine interactive system, intelligent learning systems aim to capture users' current intentions and provide recommendations through real-time feedback. However, we have a limited…

The examination of the testing mode effect has received increased attention as higher education has shifted to remote testing during the COVID-19 pandemic. We believe the testing mode effect consists of four components: the ability to physically write on the test, the method of answer recording, the proctoring/testing environment, and the effect…

ALEKS is an adaptive learning and assessment system, with courses covering subjects such as math, chemistry, and statistics. In this work, we focus on the ALEKS math courses, which cover a wide range of content starting at second grade math and continuing through college-level precalculus. To help instructors and students navigate this content,…

In Spring 2020, the need for redesigning online assessments to preserve integrity became a priority to many educators. Many of us found methods to proctor examinations using Zoom and proctoring software. Such examinations pose their own issues. To reduce the technical difficulties and cost, many Zoom proctored examination sessions were shortened;…

The rising use of technology in classrooms has also brought with it a concomitant wave of computer-based assessments. The argument for computer-based testing is often framed in terms of efficiency and data management: computer-based tests facilitate more efficient processing of test data and the rate at which feedback can be leveraged for student…

ETS scientists developed a human-centered AI (HAI) framework that combines data on how students interact with assessments--such as task navigation and time spent--with their performance, providing deeper insights into student performance in large-scale assessments.