The action sequences of respondents in problem-solving tasks reflect rich and detailed information about their performance, including differences in problem-solving ability, even if item scores are equal. It is therefore not sufficient to infer individual problem-solving skills based solely on item scores. This study is a preliminary attempt to…

In the human-to-human Collaborative Problem Solving (CPS) test, students' problem-solving process reflects the interdependency among partners. The high interdependency in CPS makes it very sensitive to group composition. For example, the group outcome might be driven by a highly competent group member, so it does not reflect all the individual…

Testing programs have explored the use of technology-enhanced items alongside traditional item types (e.g., multiple-choice and constructed-response items) as measurement evidence of latent constructs modeled with item response theory (IRT). In this report, we discuss considerations in applying IRT models to a particular type of adaptive testlet…

This paper draws on numerous data sources to better understand the shift from additive to multiplicative thinking in years 4 to 9. Research studies that have used the Scaffolding Numeracy in the Middle Years assessment tasks have found that while students can be supported to move through the early and upper zones of the Learning and Assessment…

The term speed-accuracy tradeoff is used when an increase in response speed comes at the expense of response accuracy. Although originally a concept from experimental psychology, the speed-accuracy tradeoff has been a topic in psychological assessment, too. In the first part of the manuscript, we discuss motivational factors that may be…

Multiple group modeling is one of the methods to address the measurement noninvariance issue. Traditional studies on multiple group modeling have mainly focused on item responses. In computer-based assessments, joint modeling of response times and action counts with item responses helps estimate the latent speed and action levels in addition to…

We propose a dyadic Item Response Theory (dIRT) model for measuring interactions of pairs of individuals when the responses to items represent the actions (or behaviors, perceptions, etc.) of each individual (actor) made within the context of a dyad formed with another individual (partner). Examples of its use include the assessment of…

The strategy choice model (SCM) is a highly influential theory of human problem-solving. One strength of this theory is the allowance for both item and person variance to contribute to problem-solving outcomes, but this central tenet of the model has not been empirically tested. Explanatory item response theory (EIRT) provides an ideal approach to…

The study was conducted to analyze the items about the ability of high school students diagram representation and obtain Item Curve Characteristic. Grid test instruments are compiled based on competencies and indicators of diagram representation which are then used to compile items. The test instrument consisted of five items and was validated by…

Many classroom-observation instruments have been developed (e.g., Gleason et al., 2017; Nava et al., 2019; Sawada et al., 2002), but a very small number of studies published in refereed journals have rigorously examined the quality of the ratings and the instrument using measurement models. For example, Gleason et al. developed a mathematics…

The present study examines the features and quality of visual representations (VRs) created by middle school students with learning disabilities and difficulties in mathematics in response to a released fraction item from the National Assessment of Educational Progress (NAEP). Relations between VR quality and scores on other measures of fraction…

The strategy choice model (SCM) is a highly influential theory of human problem-solving. One strength of this theory is the allowance for both item and person variance to contribute to problem-solving outcomes, but this central tenet of the model has not been empirically tested. Explanatory item response theory (EIRT) provides an ideal approach to…

This study focused on the effect of examinees' ability levels on the relationship between Reflective-Impulsive (RI) cognitive style and item response time in computerized adaptive testing (CAT). The total of 56 students majoring in Educational Technology from Shandong Normal University participated in this study, and their RI cognitive styles were…

This paper attempts to quantify the accuracy limit of "nextitem-correct" prediction by using numerical optimization to estimate the student's probability of getting each question correct given a complete sequence of item responses. This optimization is performed without an explicit parameterized model of student behavior, but with the…

Detection of students' ability levels is one of the common aims in educational studies. Cognitive Diagnosis Modeling approach has been used recently for the purpose of ability level detection by defined Q-matrices. To evaluate students' strengths and weaknesses, determine their mastery skills, and design instructions and interventions in learning…