This study aimed to develop a K-12 classroom observation protocol to assess K-12 teachers' implementation of science, technology, engineering, and mathematics (STEM) integration. The intended purpose of the observation protocol is for researchers to examine how K-12 teachers implement the STEM integrated curriculum. Based on research on STEM…

Even when raters demonstrate agreement in the use of a measure, limited score variability or violation of often-ignored statistical assumptions can result in lower reliability estimates than intuitively expected. This article uses data drawn from two randomized controlled trials of schema therapy and cognitive behavioral therapy for the treatment…

Transparency is once again a central issue of debate across types of qualitative research. Work on how to conduct qualitative data analysis, on the other hand, walks us through the step-by-step process on how to code and understand the data we've collected. Although there are a few exceptions, less focus is on transparency regarding…

Mixed-format assessments made up of multiple-choice (MC) items and constructed response (CR) items that are scored using rater judgments include unique psychometric considerations. When these item types are combined to estimate examinee achievement, information about the psychometric quality of each component can depend on that of the other. For…

Background: Measures that assess muscle strength and its development, either voluntarily or involuntarily, are important in the clinical and research context. The main aim of this study was to verify the interday reliability and the minimum detectable change (MDC) of the knee extensors muscles torque using evoked contractions and explosive…

Testing for differential item functioning (DIF) has undergone rapid statistical developments recently. Moderated nonlinear factor analysis (MNLFA) allows for simultaneous testing of DIF among multiple categorical and continuous covariates (e.g., sex, age, ethnicity, etc.), and regularization has shown promising results for identifying DIF among…

Understanding the intraindividual relation between an individual's speed and ability in testing scenarios is essential to assure a fair assessment. Different approaches exist for estimating this relationship, that either rely on specific study designs or on specific assumptions. This paper aims to add to the toolbox of approaches for estimating…

The current study evaluated a brief, informant-based autism interview: the Developmental, Dimensional and Diagnostic Interview -- Adult Version (3Di-Adult). Feasibility, reliability and validity of the Dutch 3Di-Adult was tested amongst autistic participants (n = 62) and a non-autistic comparison group (n = 30) in the Netherlands. The 3Di-Adult…

Notwithstanding the neuromyth controversy, the malleability of learning style preferences impacts the validity of the measurement instrument and the effectiveness of the associated model of learning. This study investigates the test-retest reliability and underlying dynamics of Kolb's Learning Style Inventory (KLSI). It surveys 245 college-level…

A slowdown or a speedup in response times across experimental conditions can be taken as evidence of online deployment of knowledge. However, response-time difference measures are rarely evaluated on their reliability, and there is no standard practice to estimate it. In this article, we used three open data sets to explore an approach to…

We describe best practices in reporting reliability estimates in counseling research with consideration to precision, generalization, and diverse populations. We provide a historical context to reporting reliability estimates, the limitations of past practices, and new methods to address reliability generalization. We highlight best practices…

Intercoder reliability can increase trustworthiness, accuracy, rigor, collaboration, and power sharing in qualitative research. Though not every qualitative design can utilize intercoder reliability, this article highlights how positivist qualitative research, community-based participatory research, and participatory evaluation all strengthen when…

This study aimed to adapt and validate the Parents Education Anxiety Questionnaire (PEAQ) for the Indonesian context. The sample included 222 parents of school-aged children, predominantly mothers (84.7%) and fathers (15.3%). The results indicated that the adapted questionnaire exhibited good reliability, with a Cronbach's alpha coefficient of…

Procedural fidelity refers to the degree to which procedures for an assessment or intervention (i.e., independent variables) are implemented consistent with the prescribed protocols. Procedural fidelity is an important factor in demonstrating the internal validity of an experiment and clinical treatments. Previous reviews evaluating the inclusion…

Numerous methods exist to determine the optimal number of classes when using latent profile analysis (LPA), but none are consistently correct. Recently, the likelihood incremental percentage per parameter (LI3P) was proposed as a model effect-size measure. To evaluate the LI3P more thoroughly, we simulated 50,000 datasets, manipulating factors…