This study describes the development and validation process of a computational thinking (CT) test for adults. The team designed a set of items and explored a subset of those through two qualitative pilots. Then, in order to provide validity evidence based on the test content, a team of 11 subject-matter experts coded the initial pool of items…

Computational thinking (CT) is an emerging and multifaceted competence important to the computing era. However, despite the growing consensus that CT is a competence domain, its theoretical and empirical account remain scarce in the current literature. To address this issue, rigorous psychometric evaluation procedures were adopted to investigate…

The purpose of this study was to examine the psychometric properties of the Precalculus Concept Assessment (PCA), a 25-item multiple-choice instrument designed to assess student reasoning abilities and understanding of foundational calculus concepts (Carlson et al., 2010). When this study was conducted, the extant research on the PCA and the PCA…

Technological advancements, particularly in artificial intelligence, significantly transform our society and work practices. Computational thinking (CT) has emerged as a crucial 21st-century skill, enabling individuals to solve problems more effectively through an automation-oriented perspective and fundamental concepts of computer science. To…

This methodological paper examines current conceptions of reliability in chemistry education research (CER) and provides recommendations for moving beyond the current reliance on reporting coefficient alpha (a) as reliability evidence without regard to its appropriateness for the research context. To help foster a better understanding of…

Component Universe Score Profile analysis (CUSP) is introduced in this paper as a psychometric alternative to multivariate profile analysis. The theoretical foundations of CUSP analysis are reviewed, which include multivariate generalizability theory and constrained principal components analysis. Because CUSP is a combination of generalizability…

The subject of this report is a pair of written, group-administered tests designed to measure the performance of grade 1 and grade 2 students at the beginning of the school year in the domain of number and operations. These tests build on previous versions field-tested in fall 2013 (Schoen, LaVenia, Bauduin, & Farina, 2016). Because the tests…

Bifactor latent structures were introduced over 70 years ago, but only recently has bifactor modeling been rediscovered as an effective approach to modeling "construct-relevant" multidimensionality in a set of ordered categorical item responses. I begin by describing the Schmid-Leiman bifactor procedure (Schmid & Leiman, 1957) and highlight its…

Using multitrait, multimethod data, and confirmatory factor analysis, the current study examined the effects of arithmetic item formatting and the possibility that across formats, abilities other than arithmetic may contribute to children's answers. Measurement hypotheses were guided by several leading theories of arithmetic cognition. With a…

Aim: Children and adolescents highly value their ability to participate in relevant daily life and recreational activities. The Activities Scale for Kids-performance (ASKp) instrument measures the frequency of performance of 30 common childhood activities, and has been shown to be valid and reliable. A revised and expanded 38-item ASKp (ASKp38)…

Item factor analysis has a rich tradition in both the structural equation modeling and item response theory frameworks. The goal of this paper is to demonstrate a novel combination of various Markov chain Monte Carlo (MCMC) estimation routines to estimate parameters of a wide variety of confirmatory item factor analysis models. Further, I show…

Emphasis on regular mathematics skill assessment, intervention, and progress monitoring under the RTI model has created a need for the development of assessment instruments that are psychometrically sound, reliable, universal, and brief. Important factors to consider when developing or selecting assessments for the school environment include what…

The aim of this study was to validate an individualised measure of quality of life (WeDQoL-Goals-Thailand). Three hundred and sixty-nine Thai people completed the WeDQoL by interview. Respondents rated (0-2) the perceived necessity for wellbeing of 51 goals (goal necessity), then rated (0-3) their satisfaction with the same goals (goal…

The impact of outliers on Cronbach's coefficient [alpha] has not been documented in the psychometric or statistical literature. This is an important gap because coefficient [alpha] is the most widely used measurement statistic in all of the social, educational, and health sciences. The impact of outliers on coefficient [alpha] is investigated for…

This article describes a model for response times that is proposed as a supplement to the usual factor-analytic model for responses to graded or more continuous typical-response items. The use of the proposed model together with the factor model provides additional information about the respondent and can potentially increase the accuracy of the…