This article provides a process to carefully evaluate the suitability of a content domain for which diagnostic classification models (DCMs) could be applicable and then optimized steps for constructing a test blueprint for applying DCMs and a real-life example illustrating this process. The content domains were carefully evaluated using a set of…

Providing diagnostic feedback about growth is crucial to formative decisions such as targeted remedial instructions or interventions. This article proposed a longitudinal higher-order diagnostic classification modeling approach for measuring growth. The new modeling approach is able to provide quantitative values of overall and individual growth…

Science education is crucial for fostering knowledge across academic disciplines. Past efforts to enhance science achievement at the elementary level have explored various instructional strategies. Among these, the Understanding by Design (UbD) model has shown notable potential in improving science achievement outcomes compared to traditional…

Multiple choice questions are traditionally expensive to produce. Recent advances in large language models (LLMs) have led to fine-tuned LLMs that generate questions competitive with human-authored questions. However, the relative capabilities of ChatGPT-family models have not yet been established for this task. We present a carefully-controlled…

This study aimed to examine authentic models of science assessment in assessing the competence of senior high school students who met the criteria of validity, practicality, and effectiveness. The methodology applied was an evaluation and developmental research. Research was conducted in senior high school in accordance with the needs of the…

A psychological framework for different types of items commonly used with mixed-format exams is proposed. A choice model based on signal detection theory (SDT) is used for multiple-choice (MC) items, whereas an item response theory (IRT) model is used for open-ended (OE) items. The SDT and IRT models are shown to share a common conceptualization…

Learning about the Earth through geoscience education is important in order to make informed decisions about the future of the Earth. The aim of this study is to examine students' conceptual development on geoscience subjects based on inquiry-based learning. 7th grade students participated in this study. The researcher prepared lesson plans and…

The Next Generation Science Standards (NGSS) delineate three interrelated dimensions that describe what students should know and how they should engage in science learning. These present significant challenges for assessment because traditional assessments may not be able to capture the ways in which students engage with content. Science…

This work is the fourth of a series of papers applying multidimensional item response theory (MIRT) to widely used physics conceptual assessments. This study applies MIRT analysis using both exploratory and confirmatory methods to the Brief Electricity and Magnetism Assessment (BEMA) to explore the assessment's structure and to determine a…

Background: Formative assessments are needed to enable monitoring how student knowledge develops throughout a unit. Constructed response items which require learners to formulate their own free-text responses are well suited for testing their active knowledge. However, assessing such constructed responses in an automated fashion is a complex task…

In this article, we extend Liao's test for across-group comparisons of the fixed effects from the generalized linear model to the fixed and random effects of the generalized linear mixed model (GLMM). Using as our basis the Wald statistic, we developed an asymptotic test statistic for across-group comparisons of these effects. The test can be…

Large scale international assessments depend on invariance of measurement across countries. An important consideration when observing cross-national differential item functioning (DIF) is whether the DIF actually reflects a source of bias, or might instead be a methodological artifact reflecting item response theory (IRT) model misspecification.…

In a signal detection theory (SDT) approach to multiple choice exams, examinees are viewed as choosing, for each item, the alternative that is perceived as being the most plausible, with perceived plausibility depending in part on whether or not an item is known. The SDT model is a process model and provides measures of item difficulty, item…

Knowledge of embryology is foundational for understanding normal anatomy and birth defects, yet, embryology is a notoriously difficult subject for medical students. Embryonic lateral folding in particular is one of the most challenging concepts in embryology. Highly effective teaching methods that promote active engagement with dynamic,…

Understanding whether or not different types of students master various attributes can aid future learning remediation. In this study, two-level diagnostic classification models (DCMs) were developed to represent the probabilistic relationship between external latent classes and attribute mastery patterns. Furthermore, variational Bayesian (VB)…