Assessing students' (mis)conceptions is a challenging task for teachers as well as for researchers. While individual assessment, for example through interviews, can provide deep insights into students' thinking, this is very time-consuming and therefore not feasible for whole classes or even larger settings. For those settings, automatically…

The National Assessment of Educational Progress (NAEP), often referred to as The Nation's Report Card, offers a window into the state of U.S. K-12 education system. Since 2017, NAEP has transitioned to digital assessments, opening new research opportunities that were previously impossible. Process data tracks students' interactions with the…

Changepoints are abrupt variations in a sequence of data in statistical inference. In educational and psychological assessments, it is pivotal to properly differentiate examinees' aberrant behaviors from solution behavior to ensure test reliability and validity. In this paper, we propose a sequential Bayesian changepoint detection algorithm to…

This study implemented the principles of dynamic assessment (DA) with computer technology, iSpring Quiz Maker, to (1) identify the English listening difficulties of 172 L2 English learners; (2) diagnose their individual learning needs, and (3) promote their future potential abilities. Upon evaluating the participating junior high school students'…

This study aims to develop a score equating application for computer-based school exams using parallel test kits with 25% anchor items. The items are arranged according to HOTS (High Order Thinking Skill) category, and use a scientific approach according to the physics lessons characteristics. Therefore, the questions were made using stimulus,…

The aim of the present study is two-fold. First, it uses eye-tracking to investigate the dynamics of item reading, both in multiple choice and matching items, before and during two hearings of listening passages in a computerized while-listening performance (WLP) test. Second, it investigates answer changing during the two hearings, which include…

This study investigated the performance of four widely used data-collection designs in detecting test-mode effects (i.e., computer-based versus paper-based testing). The experimental conditions included four data-collection designs, two test-administration modes, and the availability of an anchor assessment. The test-level and item-level results…

Technology-enhanced items (TEIs) are innovative, computer-delivered test items that allow test takers to better interact with the test environment compared to traditional multiple-choice items (MCIs). The interactive nature of TEIs offer improved construct coverage compared with MCIs but little research exists regarding students' performance on…

We examined semiotic exchangeability in pop-up glossary translations and illustrations used as supports for second language learners (SLLs) in computer-administered mathematics tests. In a sample of 516 mathematics items, Grades 3-8 and 11, from a large-scale assessment program in the US, test developers identified terms that could be translated…

Substantial progress has been made toward applying technology enhanced conversation-based assessments (CBAs) to measure the English-language proficiency of English learners (ELs). CBAs are conversation-based systems that use conversations among computer-animated agents and a test taker. We expanded the design and capability of prior…

This paper provides the National Center for Education Statistics (NCES), National Assessment Governing Board (NAGB), and the National Assessment of Educational Progress (NAEP) community with information that may help maintain the validity and utility of the NAEP assessments for civics and U.S. history as revisions are planned to the NAEP…

Our study is on developing mathematics items for Computer-Based Testing (CBT) using Tablet PC. These items are subject-based items using interactive dynamic objects. The purpose of this study is to obtain some suggestions for further tasks drawing on field test results for developed items. First, we clarified the role of the interactive dynamic…

Language proficiency testing serves an important function of classifying examinees into different categories of ability. However, misclassification is to some extent inevitable and may have important consequences for stakeholders. Recent research suggests that classification efficacy may be enhanced substantially using computerized adaptive…

The Programme for International Student Assessment (PISA) is an important cross-national study of 15-year-olds' academic knowledge and skills. Educationalists and public policymakers eagerly await the tri-annual results, with particular interest in whether their country has moved up or slid down the international rankings, as compared to earlier…

The work we present in this paper aims to help teachers create multiple-choice science tests. We focus on a scientific vocabulary-learning scenario taking place in a Basque-language educational environment. In this particular scenario, we explore the option of automatically generating Multiple-Choice Questions (MCQ) by means of Natural Language…