This study examined the widely used threshold of 0.2 for Yen's Q3, an index for violations of local independence. Specifically, a simulation was conducted to investigate whether Q3 values were related to the magnitude of bias in estimates of reliability, item parameters, and examinee ability. Results showed that Q3 values below the typical cut-off…

This study reviews existing research on the use of large language models (LLMs) for automatic item generation (AIG). We performed a comprehensive literature search across seven research databases, selected studies based on predefined criteria, and summarized 60 relevant studies that employed LLMs in the AIG process. We identified the most commonly…

This study aims to determine the reliability of the Beck Depression Inventory (BDI), a widely used tool for diagnosing depression--a condition that significantly impacts individuals' lives--through meta-analysis, an advanced statistical technique. To achieve this objective, studies conducted in Turkey between 1961 and 2021 that utilized the Beck…

Metacognitive awareness is essential in regulating second language (L2) listening and has been predominantly assessed by a multidimensional instrument named the Metacognitive Awareness Listening Questionnaire (MALQ). Since previous studies have yielded inconclusive evidence concerning the generalization of MALQ, it is important to examine the…

The ever-changing advances in technology require digital literacy skills for success in the workplace. To determine the critical digital literacy skills needed in the workplace today, the development of a reliable, valid instrument occurred using the nine steps of scale development by DeVellis and Thorpe (2021). Based on the SkillRise (2020a)…

While multimodality and multiliteracies has been a concept for 25 years (Kalantzis & Cope, 2023; The New London Group, 1996), research on and application of the concept within text complexity measures has been limited. Attempts to assess multiliteracies and multimodality (Jacobs, 2013; Schmerbeck & Lucht, 2017; Wyatt-Smith & Kimber,…

Despite the evolution of the grit conceptualization over the years, the psychometric validity of scales and construct structure remain unclear. Consequently, this study aims to provide new evidence that broadens the current understanding of the grit's dimensional nature in the Spanish population by examining the psychometric properties of the…

The author provides information about reliability and areas that educators should examine in determining if an assessment is consistent and trustworthy for use, and how it should be interpreted in making decisions about students. Reliability areas that are discussed in the column include internal consistency, test-retest or stability, inter-scorer…

This study describes the development of the science, technology, engineering, and mathematics (STEM) Scale, intended to assess parental attitudes toward school programs designed to deliver STEM, and evaluates its psychometric properties. The study group included 400 parents of students (138 males and 262 females) enrolled in STEM programs…

This study adapts the "Scales for Identifying Gifted Students (SIGS-2)" into Turkish for use from preschool onward, specifically during the candidate nomination stage. Conducted with 974 parents (675 mothers, 299 fathers) of children aged 5-10, it employs Confirmatory Factor Analysis (CFA) to evaluate the scale's structure and…

The study aimed to analyze, synthesize, and investigate the measurement properties of clinical tests that assess foot posture in children and adolescents. The study included research published in scientific journals that analyzed the measurement properties of clinical tests, focusing on the validity, reliability, responsiveness, or specificity of…

The "LFK" index has been promoted as an improved method to detect bias in meta-analysis. Putatively, its performance does not depend on the number of studies in the meta-analysis. We conducted a simulation study, comparing the "LFK" index test to three standard tests for funnel plot asymmetry in settings with smaller or larger…

In the early years, it has become essential to support the acquisition of computational thinking, which is seen as a 21st-century skill and new literacy. A valid and reliable measurement tool is needed to develop and evaluate educational practices related to these skills. "TechCheck" is a validated unplugged assessment of computational…

The authors contend that children benefit from play as a form of intervention and as a means of fostering their cognitive, social, and physical growth. They review several standardized instruments developed over the last fifty years to assess this benefit of play on child development. They identify twenty-one such play measures, the majority of…

We discuss an application of Bayesian factor analysis for estimation of the optimal linear combination and associated maximal reliability of a multi-component measuring instrument. The described procedure yields point and credibility interval estimates of this reliability coefficient, which are readily obtained in educational and behavioral…