We describe best practices in reporting reliability estimates in counseling research with consideration to precision, generalization, and diverse populations. We provide a historical context to reporting reliability estimates, the limitations of past practices, and new methods to address reliability generalization. We highlight best practices…

As evidence from evaluation and experimental studies continue to influence decision and policymaking, applied researchers and practitioners require tools to derive valid and credible inferences. Over the past several decades, research in causal inference has progressed with the development and application of propensity scores. Since their…

This study aimed to provide meta-analytic reliability information of the Columbia-Suicide Severity Rating Scale (C-SSRS). We implemented systematic search procedures to 35 eligible studies (N = 23,247; Mage = 26.74 years) that reported reliability estimates. The synthesized average values of Cronbach's alpha were 0.88 (95% CI [0.85, 0.92]) for the…

Objective: Brief Self-Control Scale (BSCS) is the most widely used instrument to assess self-control. The purpose of this reliability generalization meta-analysis was to examine the degree to which consistency reliability coefficients for scores on the BSCS generalize across age groups and languages. Method: We included studies using the BSCS and…

Employing G-theory and rater interviews, the study investigated how a high-stakes writing assessment procedure (i.e., a single-task, single-rater, and holistic scoring procedure) impacted the variability and reliability of its scores within the Turkish higher education context. Thirty-two essays written on two different writing tasks (i.e.,…

The ACTFL Assessment of Performance Toward Proficiency in Languages (AAPPL, https://www.actfl.n.d.org/assessments/k-12-assessments/aappl) assesses proficiency in 11 languages for students in grades 3 to 12 and is often used to award the Seal of Biliteracy. While arguments for the valid interpretation and uses of the AAPPL have previously been…

The digital transformation in educational assessment has led to the proliferation of large-scale data, offering unprecedented opportunities to enhance language learning, and testing through machine learning (ML) techniques. Drawing on the extensive data generated by online English language assessments, this dissertation investigates the efficacy…