The increasing use of computerization in the testing industry and the need for items potentially measuring higher-order skills have led educational measurement communities to develop technology-enhanced (TE) items and conduct validity studies on the use of TE items. Parallel to this goal, the purpose of this study was to collect validity evidence…

A psychological framework for different types of items commonly used with mixed-format exams is proposed. A choice model based on signal detection theory (SDT) is used for multiple-choice (MC) items, whereas an item response theory (IRT) model is used for open-ended (OE) items. The SDT and IRT models are shown to share a common conceptualization…

Technology-Enhanced Items (TEIs) have been purported to be more motivating and engaging to test takers than traditional multiple-choice items. The claim of enhanced engagement, however, has thus far received limited research attention. This study examined the rates of rapid-guessing behavior received by three types of items (multiple-choice,…

This study aims to demonstrate the optimal way to determine the cut-off score to be used to interpret the total scores obtained from an achievement test or scale using the Artificial Neural Networks method. To this end, the multiple-choice item responses in the Booklet-11 Mathematics subtest at the 8th grade level in the TIMSS 2015 Turkey sample…

In recent years, Automatic Item Generation (AIG) has increasingly shifted from theoretical research to operational implementation, a shift raising some unforeseen practical challenges. Specifically, generating high-quality answer choices presents several challenges such as ensuring that answer choices blend in nicely together for all possible item…

In this paper, we analyse a large, opportunistic dataset of responses (N = 219,826) to online, diagnostic multiple-choice mathematics questions, provided by 6-16-year-old UK school mathematics students (N = 7302). For each response, students were invited to indicate on a 5-point Likert-type scale how confident they were that their response was…

Design-based learning has been recognized by educational scholars as the key approach to science, technology, engineering, and mathematics (STEM) education at K-12 levels. However, it is unclear whether, and which dimensions of, design thinking mindsets support the conceptual learning of science. This quasi-experimental study aims to explore 37…

The Performance Evaluation for Alaska's Schools (PEAKS) assessment is administered annually statewide to students in grades 3 through 9 in ELA and mathematics. It provides students the opportunity to show their understanding of "Alaska's English Language Arts (ELA) and Mathematics Standards." The assessments provide information to…

This three-paper dissertation explores item cluster-based assessments, first in general as it relates to modeling, and then, specific issues surrounding a particular item cluster-based assessment designed. There should be a reasonable analogy between the structure of a psychometric model and the cognitive theory that the assessment is based upon.…

As part of integrated instruction, educators aim to efficiently and simultaneously address learning goals across multiple subject areas like English language arts (ELA) and science or social studies. While studies are beginning to show that this instructional approach can be effective, it is challenging to implement in practice, requiring that…

Background: Poor mathematics performance in South Africa is well known. The COVID-19 pandemic was expected to exacerbate the situation. Aim: To investigate Grade 7 learners' mathematical knowledge at the end of primary school and to compare mathematical performance of Grade 7 and 8 learners in the context of the pandemic. Setting: Data were…

Despite the increasing number of U.S. born Latinos, placing heritage and native speakers in the Spanish curriculum is still a challenge (MacGregor-Mendoza & Moreno, 2020). The present article: (1) addresses the unique needs of heritage speakers in the Spanish curriculum; (2) problematizes traditional grammar-based placement exams; and (3)…

Using simulated data, this study examined the impact of different levels of stringency of the valid case inclusion criterion on item response theory (IRT)-based true score equating over 5 years in the context of K-12 assessment when growth in student achievement is expected. Findings indicate that the use of the most stringent inclusion criterion…

In recent years, computerized adaptive testing (CAT) has gained popularity as an important means to evaluate students' ability. Assigning tags to test questions is crucial in CAT. Manual tagging is widely used for constructing question banks; however, this approach is time-consuming and might lead to consistency issues. Automatic question tagging,…

In the test-centric Thai education system, results on national exams are often viewed as indicators of educational success. These exams use multiple-choice which can have detrimental effects on students' attitudes and learning. If school assessments also rely on multiple-choice exams, the situation would be worrying, yet there is little data…