Traditional IRT and IRTree models are not appropriate for analyzing the item that simultaneously consists of multiple-choice (MC) task and constructed-response (CR) task in one item. To address this issue, this study proposed an item response tree model (called as IRTree-MR) to accommodate items that contain different response types at different…

Given the widespread presence of competition in educational settings and its complex impact on science learning, how students complete tasks in competitive environments has attracted a wide range of attention. The purpose of this study is to compare students' performance on physics conceptual questions in noncompetitive and competitive…

The intricate relationship between representational competence and content knowledge in organic chemistry has been widely debated, and the ways in which representations contribute to task difficulty, particularly in assessment, remain unclear. This paper presents a multiple-choice test instrument for assessing individuals' knowledge of fundamental…

Introduction: Students often answer questions from available expository texts for assessment and learning purposes. These activities require readers to activate not only meaning-making processes (e.g., paraphrases or elaborations), but also metacognitive operations (e.g., monitoring readers' own comprehension or self-regulating reading behaviors)…

In the present study, we recorded the eye movements of 20 criterion-based selected trainee teachers in economics while they responded to 25 single choice (SC) items in an economic content knowledge (CK) test and rated their confidence for each response in a digital assessment. By using a multilevel modeling approach with crossed random effects, we…

Prior research has emphasized that two-tier tests are an effective approach to guiding students through game tasks; furthermore, self-efficacy, which is potentially related to students' surroundings and academic performance, has a positive impact on students' cognitive outcomes in games. This implies the significance of conducting a study to probe…

Multiple-choice quizzes (MCQs) are a popular form of assessment. A rapid shift to online assessment during the COVID-19 pandemic in 2020, drove the uptake of MCQs, yet limited invigilation and wide access to material on the internet allow students to solve the questions via internet search. ChatGPT, an artificial intelligence (AI) agent trained on…

Answering the call for response process-based validation, this study shows how researchers can evaluate validity evidence of test scores based on the mental "processes" test-takers use, rather than based on correlations with other "outcome" measures. The proposed methods for process-based validation studies are demonstrated…

Three-dimensional (3D) digital anatomical models show potential to demonstrate complex anatomical relationships; however, the literature is inconsistent as to whether they are effective in improving the anatomy performance, particularly for students with low spatial visualization ability (Vz). This study investigated the educational effectiveness…

Digital games have been used in various disciplines to enhance students' learning interest and effectiveness through the gaming contexts. However, most of the digital educational games use multiple-choice questions to confirm students' learning status, implying the challenge of understanding the actual learning status of students. Also, the gaming…

Eye-tracking technology can create a record of the location and duration of visual fixations as a test-taker reads test questions. Although the cognitive process the test-taker is using cannot be directly observed, eye-tracking data can support inferences about these unobserved cognitive processes. This type of information has the potential to…

The development and evaluation of valid assessments of scientific reasoning are an integral part of research in science education. In the present study, we used the linear logistic test model (LLTM) to analyze how item features related to text complexity and the presence of visual representations influence the overall item difficulty of an…

The current study examined the relationship between test-taker cognition and psychometric item properties in multiple-selection multiple-choice and grid items. In a study with content-equivalent mathematics items in alternative item formats, adult participants' tendency to respond to an item was affected by the presence of a grid and variations of…

Taking a test improves memory for that tested information, a finding referred to as the testing effect. Multiple-choice tests tend to produce smaller testing effects than do cued-recall tests, and this result is largely attributed to the different processing that the two formats are assumed to induce. Specifically, it is generally assumed that the…

The use of concurrent data elicitation procedures (e.g. think-alouds, eye-tracking, response time) to investigate learners' cognitive processing and processes is becoming more prominent in research designs as researchers seek to acquire a better understanding of how second language (L2) learners process L2 data (e.g. Martin et al., 2019; Rogers,…