This study examined the diagnostic accuracy associated with decision making as is typically conducted with curriculum-based measurement (CBM) approaches to progress monitoring. Using previously published estimates of the standard errors of estimate associated with CBM, 20,000 progress-monitoring data sets were simulated to model student reading…

As educational practitioners and policymakers expand the range of student outcomes they assess, student perception surveys--particularly those targeting social-emotional learning--have grown in popularity. Despite excitement around the potential for measuring a wider array of important student outcomes, concerns about the validity of the…

Language in the Individuals With Disabilities Education Improvement Act (IDEIA) allows the use of response-to-intervention (RTI) methodology in the identification of specific learning disabilities. However, there is no consensus on decision rules using curriculum-based measurement of oral reading fluency (CBM-R) for defining responsiveness. The…

Best practice in gifted and talented identification procedures involves making decisions on the basis of multiple measures. However, very little research has investigated the impact of different methods of combining multiple measures. This article examines the consequences of the conjunctive ("and"), disjunctive/complementary…

Personnel selection shows an enduring need for short stand-alone tests consisting of, say, 5 to 15 items. Despite their efficiency, short tests are more vulnerable to measurement error than longer test versions. Consequently, the question arises to what extent reducing test length deteriorates decision quality due to increased impact of…

Introduction: The reliability, minimal detectable change (MDC), and construct validity of the evidence-based practice confidence (EPIC) scale were evaluated among physical therapists (PTs) in clinical practice. Methods: A longitudinal mail survey was conducted. Internal consistency and test-retest reliability were estimated using Cronbach's alpha…

A teacher's value-added score is intended to convey how much that teacher has contributed to student learning in a particular subject in a particular year. Different school districts define and compute value-added scores in different ways. A variety of people may see value-added estimates, and each group may use them for different purposes.…

The purpose of this study was to examine the impact of probe variability on the ability to replicate results in brief experimental analysis (BEA) of reading. In the first phase of the study, 41 first- and second- grade students completed 16 oral reading fluency probes. Calculations of probe difficulty were used to identify Low and High Variability…

Within a response to intervention model, educators increasingly use progress monitoring (PM) to support medium- to high-stakes decisions for individual students. For PM to serve these more demanding decisions requires more careful consideration of measurement error. That error should be calculated within a fixed linear regression model rather than…

We are currently at an exciting juncture in developing effective means for assessing so-called 21st-century skills in an innovative yet reliable fashion. One of these avenues leads through the world of "epistemic games" (Shaffer, 2006a), which are games designed to give learners the rich experience of professional practica within a discipline.…

Data Envelopment Analysis has become a popular tool for evaluating the efficiency of decision making units. The nonparametric approach has been widely applied to educational production. The approach is, however, deterministic and leads to biased estimates of performance in the presence of measurement error. Numerous simulation studies confirm the…

Short tests containing at most 15 items are used in clinical and health psychology, medicine, and psychiatry for making decisions about patients. Because short tests have large measurement error, the authors ask whether they are reliable enough for classifying patients into a treatment and a nontreatment group. For a given certainty level,…

Procedures are described for studying the reliability of decisions based on specific passing scores with tests made up of discrete items and designed to measure continuous rather than categorical traits. These procedures are based on the estimation of the joint distribution of true scores and observed scores. (CTM)

Studied the amount of measurement error in a state's performance-based writing task as it relates to high-stakes decision reproducibility. Using 175 eighth-grade writing samples, the study finds moderate correlations between the two raters' scores, with significant differences for the rates for the handwritten, but not the typed, essays.(SLD)