The second version of Kaiser's Measure of Sampling Adequacy (MSA[subscript 2]) has been widely applied to assess the factorability of data in psychological research. The MSA[subscript 2] is developed in the population and little is known about its behavior in finite samples. If estimated MSA[subscript 2]s are biased due to sampling errors,…

When it is required to estimate item parameters of a large item bank, Multiple Matrix Sampling (MMS) design provides an efficient way while minimizing the test burden on students. The current study exemplifies how to calibrate a large item pool using MMS design for various purposes, such as developing a CAT administration. The purpose of the…

In sample survey analysis, accurate population mean estimation is an important task, but traditional approaches frequently ignore the intricacies of real-world data, leading to biassed results. In order to handle uncertainties, indeterminacies, and ambiguity, this work presents an innovative approach based on neutrosophic statistics. We proposed…

Education policy research aimed at eliminating racism necessitates methodological innovation that fosters both equity-centered approaches and robust empirical analysis of the systemic nature of racism. Most quantitative research in educational psychology omits the racist environment that students in K-12 education exist in (DeCuir-Gunby &…

Since 2018, colleagues from two research centers at Dublin City University have been collaborating to develop a measurement scale to assess individuals' ability to identify workplace bullying. Having agreed on an operational definition of the construct, an item pool of 26 workplace bullying scenarios, that is, short descriptions of…

Many methods are used to measure interrater reliability for studies where each target receives ratings by a different set of judges. The purpose of this study is to explore the use of hierarchical modeling for estimating interrater reliability using the intraclass correlation coefficient. This study provides a description of how the ICC can be…

Recent years have seen a growing number of studies investigating the accuracy of nonprobability online panels; however, response quality in nonprobability online panels has not yet received much attention. To fill this gap, we investigate response quality in a comprehensive study of seven nonprobability online panels and three probability-based…

Purpose: Language sample analysis (LSA) represents an ecologically valid method for diagnosing, identifying goals, and measuring progress in children with developmental language disorder (DLD). LSA is, however, time consuming. The purpose of this study was to determine the length of sample needed to obtain reliable LSA measures for children in…

As the integration of digital technologies continues to shape academic landscapes, assessing digital literacy in the context of academic writing becomes paramount. Several instruments and frameworks are available for measuring digital literacy and examining it from different perspectives; however, none are suitable for measuring the digital…

Item banks are generally considered the basis of a new generation of educational measurement. In combination with specialized software, they can facilitate the computerized assembling of multiple pre-equated test forms. However, for advantages of item banks to become fully realized it is important that the item banks store a relatively large…

Successful testing programs rely on high-quality test items to produce reliable scores and defensible exams. However, determining what statistical screening criteria are most appropriate to support these goals can be daunting. This study describes and demonstrates cost-benefit analysis as an empirical approach to determining appropriate screening…

The purpose of this study was to examine scale adaptation studies in the field of science education in terms of content, method as well as to determine the general guidelines used in the scale adaptation process. The population of this study included a total of 145 journals published in the field of science education in Turkey. A total of 46…

The effect of guessing on the point estimate of coefficient alpha has been studied in the literature, but the impact of guessing and its interactions with other test characteristics on the interval estimators for coefficient alpha has not been fully investigated. This study examined the impact of guessing and its interactions with other test…

Data collected from surveys can result in hundreds of variables and thousands of respondents. This implies that time and energy must be devoted to (a) carefully entering the data into a database, (b) running preliminary analyses to identify any problems (e.g., missing data, potential outliers), (c) checking the reliability and validity of the…

A latent variable modeling approach for scale reliability evaluation in heterogeneous populations is discussed. The method can be used for point and interval estimation of reliability of multicomponent measuring instruments in populations representing mixtures of an unknown number of latent classes or subpopulations. The procedure is helpful also…