One of the challenges of discussing validity arguments for machine scoring of essays centers on the absence of a commonly held definition and theory of good writing. At best, the algorithms attempt to measure select attributes of writing and calibrate them against human ratings with the goal of accurate prediction of scores for new essays.…

Despite the increasing demand for diagnostic information, observed subscores have been often reported to lack adequate psychometric qualities such as reliability, distinctiveness, and validity. Therefore, several statistical techniques based on CTT and IRT frameworks have been proposed to improve the quality of subscores. More recently, DCM has…

This exploratory study reports on characteristics of proof production and proof writing observed in the work of first-year university students who took part in workshops on the theorem prover LEAN (https://leanprover.github.io). These workshops were voluntary and offered alongside a transition to proof module in a UK university. Through…

This systematic review explores psychometric properties of the Substance Abuse Subtle Screening Inventory adult version (SASSI-3), the most commonly used instrument by counselors for assessment and diagnosis of alcohol and substance use.

Background: Student anxiety about statistics may lead to poorer learning outcomes. Objective: The purpose of this study was to evaluate an exercise designed to teach students in an introductory statistics class the principles of bivariate regression and to emphasize how statistical tools used by psychologists are also implemented in other fields.…

Objectives: The purposes of this study were to validate the Acoustic Breathiness Index (ABI) for the Japanese-speaking population and to determine whether it is independent of factors such as sex, age, and perceptual ratings of roughness. Method: First, the concurrent validity of the ABI for perceptual breathiness was evaluated on the…

Online teaching evaluations have replaced in-class teaching evaluations as the new norm for assessing teaching performance. Faculty raise concerns regarding lower response rates compared to traditional in-class evaluations. A low response rate implies an underrepresentation of the target student population and threatens the validity of the…

Purpose: The aim of this study was to advance the use of structured, monologic discourse analysis by validating an automated scoring procedure for core lexicon (CoreLex) using transcripts. Method: Forty-nine transcripts from persons with aphasia and 48 transcripts from persons with no brain injury were retrieved from the AphasiaBank database. Five…

Cognitive diagnostic models (CDMs) have received sustained attention in educational settings because they can be used to operationalize formative assessment to provide diagnostic feedback and inform instruction. A large number of CDMs have been developed over the past few years. An important component of all CDMs is a Q-matrix that specifies a…

We report findings of a psychometric study comparing scores from two screening tools: "Student Risk Screening Scale--Internalizing and Externalizing" (SRSS-IE, an adapted version of the Student Risk Screening Scale) and "Behavior Assessment System for Children--Second Edition, Behavioral and Emotional Screening System" (BASC-2…

Students learning to read in more than one language are a growing population in schools. Early screening and identification of reading difficulties may lead to better outcomes for all students. Oral reading fluency (ORF) is one tool shown to be both a reliable measure of reading and an accurate method to identify students at risk for poor reading…

Low working memory (WM) capacity has been linked to poor academic performance and problem behavior. Availability of easy-to-administer screening tests would facilitate early detection of WM deficits. This study investigated the psychometric properties of the Working Memory Power Test for Children (WMPT) in 170 Australian schoolchildren (8½-11…

Speech fluency has been extensively researched as a core construct for second language (L2) speaking assessment. Despite the broad consensus on its multifaceted nature, few researchers have empirically explored the dimensionality of this construct. Operationalizations of fluency vary across research and practice, using both holistic and…

The purpose of this study was to evaluate the reliability, validity, and accuracy of scores from the Intervention Selection Profile--Function (ISP-Function): a brief functional assessment tool founded upon Direct Behavior Rating (DBR) methodology. Participants included 34 teacher-student dyads. Using the ISP-Function, teachers rated the extent to…

The purpose this study was to evaluate the reliability, validity, and accuracy of scores from the "Intervention Selection Profile--Function (ISP-Function)": a brief functional assessment tool founded upon Direct Behavior Rating (DBR) methodology. Participants included 34 teacher-student dyads. Using the ISP-Function, teachers rated the…