The use of brief dyslexia rating scales is increasing given current dyslexia legislation efforts across the United States. The purpose of this article is to provide an overview of the historical context of the use of brief dyslexia rating scales, strengths, and limitations of using these measures, criteria for selecting these measures, and a…

Setting cutoff scores is one of the most common practices when using scales to aid in classification purposes. This process is usually done univariately where each optimal cutoff value is decided sequentially, subscale by subscale. While it is widely known that this process necessarily reduces the probability of "passing" such a test,…

In the Bookmark standard-setting procedure, panelists are instructed to consider what examinees know rather than what they might attain by guessing; however, because examinees sometimes do guess, the procedure includes a correction for guessing. Like other corrections for guessing, the Bookmark's correction assumes that examinees either know the…

Historically, there has been concern about students losing reading ability over extended breaks from school, commonly in the summer, but studies of this phenomenon have produced inconsistent results. We applied exploratory visual analysis of multiple datasets to examine whether students in Grades K-5 appear to lose or improve in various reading…

The purpose of this study is to introduce a method for converting scored 4-option multiple-choice (MC) items into scored 3-option MC items without re-pretesting the 3-option MC items. This study describes a six-step process for achieving this goal. Data from a professional credentialing exam was used in this study and the method was applied to 24…

Recent research suggests using multiple screening measures to identify students at risk for academic difficulties may decrease the number of students incorrectly identified as such. Gated frameworks in which students that score below a cut-score on an initial measure are assessed with a follow-up measure have been recommended. Researchers have…

An Angoff standard setting study generally yields judgments on a number of items by a number of judges (who may or may not be nested in panels). Variability associated with judges (and possibly panels) contributes error to the resulting cut score. The variability associated with items plays a more complicated role. To the extent that the mean item…

Raters must score accurately and consistently for classroom observation scores to be valid. This requires (a) a standard defining when scoring is accurate and consistent enough and (b) measuring and remediating rater performance against that standard. Current practice has focused on this second problem to the exclusion of the first. My goal here…

In this paper we describe content focused, practical procedures to establish benchmarked performance standards for educational and other tests. To implement our procedures, we identify one or more external benchmark (e.g., NAEP or PISA performance levels, college and career readiness benchmarks); link to the external benchmarks to locate those…

RAND partnered with Texas Higher Education Coordinating Board and American Institutes for Research to support the state's developmental education reform efforts through a combination of continuous improvement support and analysis of statewide data on implementation. One of the reforms examined through the study was the state's recommendation that…

Brennan noted that users of test scores often want (indeed, demand) that subscores be reported, along with total test scores, for diagnostic purposes. Haberman suggested a method based on classical test theory (CTT) to determine if subscores have added value over the total score. One way to interpret the method is that a subscore has added value…

The use of the Quality Rating and Improvement System (QRIS) to improve early childhood education program quality is based in part on assumptions that the quality of programs can be measured and that quality ratings are associated with meaningful differences in learning outcomes for children. This report reviews all of the state QRIS validation…

Classroom observations are increasingly common in education policies as a means to assess the quality of teachers and/or education programs for purposes of making high-stakes decisions. This article considers one policy, the Head Start Designation Renewal System (DRS), which involves classroom observations to assess the quality of Head Start…

The authors observe in this commentary that construct maps can help standard-setting panels to make realistic and internally consistent recommendations for performance-level descriptions (PLDs) and cut-scores, but the benefits may not be realized if policymakers do not fully understand the rationale for the recommendations provided by the…

Northwest Evaluation Association™ (NWEA™) is committed to providing partners with useful tools to help make inferences from Measures of Academic Progress® (MAP®) interim assessment scores. One important tool is the concordance table between MAP and state summative assessments. Concordance tables have been used for decades to relate scores on…