Teacher certification exams are supposed to assess if a student is likely to succeed in teaching. What if an exam seems to be inappropriate? This article is an inquiry of the New York State Content Specialty Test for Early Childhood Candidates, particularly the math section. It raises the issue of whether we are asking the right questions and…

There has been increasing interest in applying integrative data analysis (IDA) to analyze data across multiple studies to increase sample size and statistical power. Measures of a construct are frequently not consistent across studies. This article provides a tutorial on the complex decisions that occur when conducting harmonization of measures…

Tests are designed as an integral part of the teaching process, necessarily including stakeholders from the onset of preparations to grade allocation, the administration of the test, and the interpretation of the results. The process commences with selecting content to evaluate, deciding upon the skills to be tested, and to meet course objectives…

Organic chemistry is a class well known to be difficult and necessary for many careers in the sciences, and as a result, has garnered interest in researching ways to improve student learning and comprehension. One potential way involves using eye tracking techniques to understand how students visually examine questions. Organic chemistry involves…

The National Assessment of Educational Progress (NAEP), often referred to as The Nation's Report Card, offers a window into the state of U.S. K-12 education system. Since 2017, NAEP has transitioned to digital assessments, opening new research opportunities that were previously impossible. Process data tracks students' interactions with the…

More than a decade after two comprehensive examinations of the technical properties of the ClassMaps Survey (CMS), this study reexamined the structure and internal consistency of the scale and, for the first time, examined its measurement invariance across gender and school levels. Participants were 1,083 elementary and middle level students from…

Nurturing student artificial intelligence (AI) competency is crucial in the future of K-12 education. Students with strong AI competency should be able to ethically, safely, healthily, and productively integrate AI into their learning. Research on student AI competency is still in its infancy, primarily focusing on theoretical and professional…

The role of artificial intelligence (AI) in education plays a crucial role in teacher training digitalisation. Although AI has enormous potential, not much is known about how pre-service teachers perceive and utilise AI tools in professional practice. Hence, this study, guided by the Unified Theory of Acceptance and Use of Technology framework,…

The Concept Formation subtest of the Woodcock Johnson Tests of Cognitive Abilities represents a dynamic test due to continual provision of feedback from examiner to examinee. Yet, the original scoring protocol for the test largely ignores this dynamic structure. The current analysis applies a dynamic adaptation of an explanatory item response…

Learning outcomes are the first and most essential element of the curricula and the correct and rigorous determination of the learning outcomes is very important in order to ensure formal education in schools to be well-planned and to design and apply curriculums effectively. Because, the other elements of the curriculum which are content,…

Raven's Standard Progressive Matrices (Raven 1941) is a widely used 60-item long measure of general mental ability. It was recently suggested that, for situations where taking this test is too time consuming, a shorter version, comprised of only the last series of the Standard Progressive Matrices (Myszkowski and Storme 2018) could be used, while…

We analyze a 12-item version of Raven's Standard Progressive Matrices test, traditionally scored with the sum score. We discuss some important differences between assessment in practice and psychometric modelling. We demonstrate some advanced diagnostic tools in the freely available R package, dexter. We find that the first item in the test…

Kelley's Discrimination Index (DI) is a simple and robust, classical non-parametric short-cut to estimate the item discrimination power (IDP) in the practical educational settings. Unlike item-total correlation, DI can reach the ultimate values of +1 and -1, and it is stable against the outliers. Because of the computational easiness, DI is…

The responses of non-effortful test-takers may have serious consequences as non-effortful responses can impair model calibration and latent trait inferences. This article introduces a mixture model, using both response accuracy and response time information, to help differentiating non-effortful and effortful individuals, and to improve item…

In this digital ITEMS module, Dr. Brian Leventhal and Dr. Allison Ames provide an overview of "Monte Carlo simulation studies" (MCSS) in "item response theory" (IRT). MCSS are utilized for a variety of reasons, one of the most compelling being that they can be used when analytic solutions are impractical or nonexistent because…