Many education tests and psychological surveys elicit respondent views of similar constructs across scenarios (e.g., story followed by multiple choice questions) by repeating common statements across scales (one-statement-multiple-scale, OSMS). However, a respondent's earlier responses to the common statement can affect later responses to it…

Factor analysis is a statistical method to explore the relationships among observed variables and identify latent structures. It is crucial in scale development and validity analysis. Key factors affecting the accuracy of factor analysis results include the type of data, sample size, and the number of response categories. While some studies…

Among all, Cronbach's Alpha and McDonald's Omega are commonly used for reliability estimations. The alpha uses inter-item correlations while omega is based on a factor analysis result. This study uses simulated ordinal data sets to test whether the alpha and omega produce different estimates. Their performances were compared according to the…

This study developed a Metaverse Competency Scale (MCS) to measure users' proficiency in crucial metaverse skills. The scale was built on a thorough literature review covering digital avatars, immersive experiences, and decentralized value exchange. A two-phase questionnaire approach was employed. In the first phase, a pre-test questionnaire,…

Measures that accurately capture the phenomenon are critical to research and practice in group work. The vast majority of group-related measures were developed using the reflective measurement model rooted in classical test theory (CTT). Depending on the construct definition and the measure's purpose, the reflective model may not always be the…

The present study was motivated by the theory-method mismatch between heterotypic continuity (aspects of development that manifest differently across the lifespan thus cannot be measured the same way over time) and longitudinal measurement equivalence (the statistical assumption that the developmental phenomenon studied is measured on the same…

The item wording (or keying) effect is respondents' differential response style to positively and negatively worded items. Despite decades of research, the nature of the effect is still unclear. This article proposes a potential reason; namely, that the item wording effect is scale-specific, and thus findings are applicable only to a particular…

The main thesis of the present study is to use the Bayesian structural equation modeling (BSEM) methodology of establishing approximate measurement invariance (A-MI) using data from a national examination in Saudi Arabia as an alternative to not meeting strong invariance criteria. Instead, we illustrate how to account for the absence of…

In many modeling contexts, the variables in the model are linear composites of the raw items measured for each participant; for instance, regression and path analysis models rely on scale scores, and structural equation models often use parcels as indicators of latent constructs. Currently, no analytic estimation method exists to appropriately…

MOCCA is an online assessment of inferential reading comprehension for students in 3rd through 6th grades. It can be used to identify good readers and, for struggling readers, identify those who overly rely on either a Paraphrasing process or an Elaborating process when their comprehension is incorrect. Here a propensity to over-rely on…

The Test of Gross Motor Development--Second Edition (TGMD-2) is a widely used evaluation tool of children's fundamental motor skills (FMS). This study illustrates how exploratory structural equation modeling (ESEM) addresses current limitations associated with TGMD-2 factor structure. Using the normative dataset from the TGMD-2 manual, we test…

Position effects may occur in both paper--pencil tests and computerized assessments when examinees respond to the same test items located in different positions on the test. To examine position effects in large-scale assessments, previous studies often used multilevel item response models within the generalized linear mixed modeling framework.…

The present study examined the ability of content domain (algebra, geometry, rational number, probability) to classify mathematics misconceptions. The study was conducted with 1,133 students in 53 algebra and geometry classes taught by 17 teachers from three high schools and one middle school across three school districts in a Midwestern state.…

Item positions in educational assessments are often randomized across students to prevent cheating. However, if altering item positions results in any significant impact on students' performance, it may threaten the validity of test scores. Two widely used approaches for detecting position effects -- logistic regression and hierarchical…

In this study, we sought to confirm the theoretical framework underlying an Online Graduate Mentoring Scale by establishing the scale's factorial validity and reliability. Analysis of data received from doctoral students and alumni/ae of the College of Education of one large, online, accredited university reduced the initial theoretical…