Both school and district administrators use the results of standardized, large-scale tests to inform decisions about the need for, or success of, educational programs and interventions. However, test results at the school level are subject to random fluctuations due to changes in cohort, test items, and other factors outside of the school's…

We examined how raters and tasks influence measurement error in writing evaluation and how many raters and tasks are needed to reach a desirable level of 0.90 and 0.80 reliabilities for children in Grades 3 and 4. A total of 211 children (102 boys) were administered three tasks in narrative and expository genres, respectively, and their written…

We examined how raters and tasks influence measurement error in writing evaluation and how many raters and tasks are needed to reach a desirable level of 0.90 and 0.80 reliabilities for children in Grades 3 and 4. A total of 211 children (102 boys) were administered three tasks in narrative and expository genres, respectively, and their written…

An information-correction method for testlet-based tests is introduced. This method takes advantage of both generalizability theory (GT) and item response theory (IRT). The measurement error for the examinee proficiency parameter is often underestimated when a unidimensional conditional-independence IRT model is specified for a testlet dataset. By…

This study examined the number of writing samples needed to obtain a reliable estimate of young struggling writers' capabilities. It further assessed if performance in one genre was reflective of performance in other genres for these children. Second- and third-grade students (81 boys, 56 girls), who were identified as struggling writers in need…

This technical report is one in a series of five describing the reliability (test/retest and alternate form) and G-Theory/D-Study research on the easyCBM reading measures, grades 1-5. Data were gathered in the spring of 2011 from a convenience sample of students nested within classrooms at a medium-sized school district in the Pacific Northwest.…

In recent years, the regression discontinuity design (RDD) has gained widespread recognition as a quasi-experimental method that when used correctly, can produce internally valid estimates of causal effects of a treatment, a program or an intervention (hereafter referred to as treatment effects). In an RDD study, subjects or groups of subjects…

A recent NBER [National Bureau of Economic Research] working paper examines Florida's policy to retain many low-scoring third graders. The report concludes that third-grade retention has immediate positive effects on the following year's test results, but these effects fade over the next six years, with no effect on graduation. The regression…

The traditional approach to scale modified-Angoff cut scores transfers the raw cuts to an existing raw-to-scale score conversion table. Under the traditional approach, cut scores and conversion table raw scores are not only seen as interchangeable but also as originating from a common scaling process. In this article, we propose an alternative…

Despite growing use of CBM Maze in universal screening and research, little information is available regarding the number of CBM Maze probes needed for reliable decisions. The current study extends existing research on the technical adequacy of CBM Maze by investigating the number of probes and assessment durations (1-3 min) needed for reliable…

The use of oral reading fluency (ORF) passages within a Response to Intervention (RTI) framework is examined. Significant limitations within the current ORF research are discussed. The passage equivalency and readability scores for DIBELS Next, AIMSweb, and a school district's curriculum's ORF passages are evaluated using Generalizability Theory…

Test-based accountability as well as value-added asessments and much experimental and quasi-experimental research in education rely on achievement tests to measure student skills and knowledge. Yet, we know little regarding fundamental properties of these tests, an important example being the extent of measurement error and its implications for…

For educators seeking to differentiate instruction, cognitive ability tests sampling multiple content domains, including verbal, quantitative, and nonverbal reasoning, provide superior information about student strengths and weaknesses compared with unidimensional reasoning measures. However, these ability tests have not been fully evaluated with…

Inclusion of students with disabilities in district-wide and state assessments is mandated by federal regulations, and teachers sometimes play an important role in rating these students' work. In this study, trained teachers rated student proficiency in performance assessments in language arts and mathematics in third, fifth, and ninth grades. The…

Generalizability (G) theory was used with a sample of 37 third-grade students to assess the variability in words correct per minute (WCPM) scores caused by student skill and passage variability. Reliability-like coefficients and the SEM based on a specific number of assessments using different combinations of passages demonstrated how manipulating…