Automatic essay scoring (AES) is an essential educational application in natural language processing. This automated process will alleviate the burden by increasing the reliability and consistency of the assessment. With the advances in text embedding libraries and neural network models, AES systems achieved good results in terms of accuracy.…

Research has shown the importance of responsive caregiving for fostering positive development early in life; however, tools measuring these interactions are often impractical for larger scale intervention trials and in settings with resource constraints. The present study provides reliability and validity evidence from Sindh, Pakistan for a tool…

The Affordances for Motor Behavior of Schoolchildren (AMBS) is a standardized self-reporting tool comprising 73 questions, organized into seven sections aiming to assess affordances in children's regular contexts. This investigation aims to establish the reliability and validity of the results obtained from the AMBS in South Indian children. The…

Children with autism spectrum disorder benefit from early, intensive interventions to improve social communication, and parent-implemented interventions are a feasible, family-centered way to increase treatment dosage. The success of such interventions is dependent on a parent's ability to implement the strategies with fidelity. However,…

Aim: To describe the development of an intervention-specific fidelity measure and its utilization and to determine whether the newly developed Sitting Together and Reaching to Play (START-Play) intervention was implemented as intended. Also, to quantify differences between START-Play and usual early intervention (uEI) services. Method: A fidelity…

The accuracy of certain internal consistency estimators have been questioned in recent years. The present study tests the accuracy of six reliability estimators (Cronbach's alpha, omega, omega hierarchical, Revelle's omega, and greatest lower bound) in 140 simulated conditions of unidimensional continuous data with uncorrelated errors with varying…

The "Raven's 2 Progressive Matrices Clinical Edition" (Raven's 2; Raven, Rust, Chan, & Zhou, 2018), published by NCS Pearson, is an individually administered nonverbal assessment of general cognitive ability developed to measure "educative abilities," defined as the ability to think clearly and solve complex problems in…

For assessments that use different forms in different administrations, equating methods are applied to ensure comparability of scores over time. Ideally, a score scale is well maintained throughout the life of a testing program. In reality, instability of a score scale can result from a variety of causes, some are expected while others may be…

In response to the call for promoting three-dimensional science learning (NRC, 2012), researchers argue for developing assessment items that go beyond rote memorization tasks to ones that require deeper understanding and the use of reasoning that can improve science literacy. Such assessment items are usually performance-based constructed…

This study examined the degree that reliability coefficients for scores on the PTGI generalize across participant and study characteristics. Meta-analytic procedures resulted in observed and predicted mean alpha coefficients ranging from acceptable to excellent and appeared to be largely unrelated to the participant characteristics included in our…

Cohen's kappa coefficient was originally proposed for two raters only, and it later extended to an arbitrarily large number of raters to become what is known as Fleiss' generalized kappa. Fleiss' generalized kappa and its large-sample variance are still widely used by researchers and were implemented in several software packages, including, among…

This study explored the influence of different take-off thresholds on the reliability and magnitude of countermovement jump (CMJ) performance variables. Twenty-three men were tested on two separate sessions. CMJ performance variables were obtained against three external loads (0.5-30-60 kg) using three take-off thresholds: 10 N (arbitrary value of…

The objective of this study was to understand behavioral characteristics of creative physicists during their growth period, and we want to use any insight gained to help teachers and parents encourage students' creativity in their everyday life. To do this, the critical incident technique was utilized to extract behavioral traits from six…

Purpose: The field lacks consensus about preferred metrics for capturing pharyngeal residue on videofluoroscopy. We explored four different methods, namely, the visuoperceptual Eisenhuber scale and three pixel-based methods: (a) residue area divided by vallecular or pyriform sinus spatial housing ("%-Full"), (b) the Normalized Residue…

Empirical researchers using qualitative comparative analysis (QCA) can work with crisp, multivalue, and fuzzy sets. The relative advantages of crisp and multivalue sets have been discussed in the QCA literature. There has been little reflection on the more frequent decision between crisp and fuzzy sets for which there often is no theoretical…