The purpose of this instructional piece was to provide a nontechnical synthesis of common internal consistency reliability estimates used in professional counseling and in related fields. The article begins with an overview of coefficients alpha, omega, omega hierarchical, and H, with guidelines for their selection. Next, I provide recommendations…

Problem Statement: Among individuals aged 12 years or older, 14.3% (40.0 million) reporting the use of an illicit drug in the previous year. Given the prevalence of drug abuse, it is increasingly important to determine effective screening practices, treatment procedures, and best practices among various subpopulations to identify drug use-related…

Quantitative measures of systematic differences in OSCE scoring across examiners (often termed examiner stringency) can threaten the validity of examination outcomes. Such effects are usually conceptualised and operationalised based solely on checklist/domain scores in a station, and global grades are not often used in this type of analysis. In…

This study was undertaken to set a novel developmental screening test for autism spectrum disorder (ASD) using the Griffiths Scales of Child Development (Griffith III) (Green et al., 2016; Stroud et al., 2016), in order to intercept the early atypical developmental patterns indicating ASD risk in the first 3 years of age. An observational and…

Given the recent push for universal screening, it is important to take into account how well a screener identifies children at risk for reading problems as well as how screener and sample information contribute to this classification. Picking the best cut-point for a particular sample and screening goal can be challenging given that test manuals…

Given the recent push for universal screening, it is important to take into account how well a screener identifies children at risk for reading problems as well as how screener and sample information contribute to this classification. Picking the best cut-point for a particular sample and screening goal can be challenging given that test manuals…

Subjects with initially extreme observations upon remeasurement are found closer to the population mean. This tendency of observations toward the mean is called regression to the mean (RTM) and can make natural variation in repeated data look like real change. Studies, where subjects are selected on a baseline criterion, should be guarded against…

Diagnostic classification models (DCMs) are psychometric models that yield probabilistic classifications of respondents according to a set of discrete latent variables. The current study examines the recently introduced one-parameter log-linear cognitive diagnosis model (1-PLCDM), which has increased interpretability compared with general DCMs due…

A recent review found that 11% of published factor models are hierarchical models with second-order factors. However, dedicated recommendations for evaluating hierarchical model fit have yet to emerge. Traditional benchmarks like RMSEA <0.06 or CFI >0.95 are often consulted, but they were never intended to generalize to hierarchical models.…

Commonly used cut-points may misclassify physical activity (PA) in people with cystic fibrosis (CF). The aim of this study was to develop and cross-validate condition-specific cut-points in children and adolescents with CF. Thirty-five children and adolescents with CF (15 girls; 11.6 ± 2.8 years) and 28 controls (16 girls; 12.2 ± 2.7 years), had…

The most common effect size when using a multiple-group confirmatory factor analysis approach to measurement invariance is [delta]CFI and [delta]TLI with a cutoff value of 0.01. However, this recommended cutoff value may not be ubiquitously appropriate and may be of limited application for some tests (e.g., measures using dichotomous items or…

Appropriate placement into courses at postsecondary institutions is critical for the success of students in terms of retention and graduation rates. To reduce the number of students who are misplaced, using multiple measures in placing students is encouraged. However, in practice most postsecondary schools utilize only a few measures to determine…

We report findings of this replication study, examining the internalizing subscale (SRSS-I4) of the revised version of the Student Risk Screening Scale for Internalizing and Externalizing behavior (SRSS-IE 9) and the internalizing subscale of the Teacher Report Form (TRF). Using the sample from 13 elementary schools across three U.S. states with…

We aimed to identify empirically valid cut scores on the positive behavioral interventions and supports (PBIS) Tiered Fidelity Inventory (TFI) through an expert panel process known as bookmarking. The TFI is a measurement tool to evaluate the fidelity of implementation of PBIS. In the bookmark method, experts reviewed all TFI items and item scores…

Purpose: The purpose of this research is to address the need for a robust system to accurately determine a cutoff score by using the Angoff method and leveraging the Rasch infit and outfit statistics of item response theory by detecting and removing misfitting items in a test. Design/methodology/approach: Researchers in educational evaluation…