Applications of polytomous IRT models in applied fields (e.g., health, education, psychology) are abound. However, little is known about the impact of the number of categories and sample size requirements for precise parameter recovery. In a simulation study, we investigated the impact of the number of response categories and required sample size…

There is a lot of interest in providing detailed reports to schools indicating which skills pupils have mastered and which still need development -- and, more broadly, the knowledge, skills and understanding that pupils have acquired and not yet acquired. Cognitive diagnostic assessment is an approach designed to provide this kind of insight.…

My dissertation research explored responder behaviors (e.g., demonstrating response styles, carelessness, and possessing misconceptions) that compromise psychometric quality and impact the interpretation and use of assessment results. Identifying these behaviors can help researchers understand and minimize their potentially construct-irrelevant…

This study aimed to address the need for a comprehensive assessment tool to evaluate the mathematical abilities of first-grade students through cognitive diagnostic assessment (CDA). The primary challenge involved in this endeavor was to delineate the specific cognitive skills and sub-skills pertinent to first-grade mathematics (FG-M) and to…

Although Diagnostic Classification Models (DCMs) were introduced to education system decades ago, it seems that these models were not employed for the original aims upon which they had been designed. Using DCMs has been mostly common in analyzing large-scale non-diagnostic tests and these models have been rarely used in developing Cognitive…

Although much research has been conducted on the psychometric properties of cognitive diagnostic models, they are only recently being used in operational settings to provide results to examinees and other stakeholders. Using this newer class of models in practice comes with a fresh challenge for diagnostic assessment developers: effectively…

Attribute structure is an explicit way of presenting the relationship between attributes in diagnostic measurement. The specification of attribute structures directly affects the classification accuracy resulted from psychometric modeling. This study provides a conceptual framework for understanding misspecifications of attribute structures. Under…

More than three decades after their introduction, diagnostic classification models (DCM) do not seem to have been implemented in educational systems for the purposes they were devised. Most DCM research is either methodological for model development and refinement or retrofitting to existing nondiagnostic tests and, in the latter case, basically…

Critical thinking skills are the 21st-century life skills that are needed by students. However, in elementary schools, there are no instruments that are truly effective and efficient to measure critical thinking skills. This research aims to develop an open-ended question assessment instrument to measure students' critical-thinking skills, to test…

Developing a diagnostic tool within the diagnostic measurement framework is the optimal approach to obtain multidimensional and classification-based feedback on examinees. However, end users may seek to obtain diagnostic feedback from existing item responses to assessments that have been designed under either the classical test theory or item…

Few of the papers published in journals and conference proceedings on problem-based learning (PBL) are empirical studies, and most of these use self-report as the measure of PBL (Beddoes, Jesiek, & Borrego, 2010). The current study provides a theoretically derived matrix for coding and classifying PBL that was objectively applied to official…

The high rate of students taking developmental education courses suggests that many students graduate from high school unready to meet college expectations. A college readiness screener can help colleges and school districts better identify students who are not ready for college credit courses. The primary audience for this guide is leaders and…

Automatic item generation represents a relatively new but rapidly evolving research area where cognitive and psychometric theories are used to produce tests that include items generated using computer technology. Automatic item generation requires two steps. First, test development specialists create item models, which are comparable to templates…

Implications of the International Classification of Functioning, Disability and Health (ICF) on the development and use of tests in school settings are enumerated. We predict increased demand for behavioural assessments that consider a person's activities, participation and person-environment interactions, including measures that: (a) address…

Evidence-centered assessment design (ECD) explicates a transparent evidentiary argument to warrant the inferences we make from student test performance. This article describes how the vehicles for gathering student evidence--task models and test specifications--are developed. Task models, which are the basis for item development, flow directly…