Little is known empirically about intraclass correlations (ICCs) for multisite cluster randomized trial (MSCRT) designs, particularly in science education. In this study, ICCs suitable for science achievement studies using a three-level (students in schools in districts) MSCRT design that block on district are estimated and examined. Estimates of…

The motivation for this paper is the authors' recent work on several randomized control trials in which they found the primary result, which averaged across subgroups or sites, to be moderated by demographic or site characteristics. They are led to examine a distinction that the Institute of Education Sciences (IES) makes between "confirmatory"…

Some form of a short interrupted time series (ITS) is often used to evaluate state and national programs. An ITS design with a single treatment group assumes that the pretest functional form can be validly estimated and extrapolated into the postintervention period where it provides a valid counterfactual. This assumption is problematic. Ambiguous…

This paper introduces a variation on the post-test only cohort control design and addresses questions concerning both the methodological credibility and the practical utility of employing this design variation in evaluations of large-scale complex professional development programmes in mathematics education. The original design and design…

In 2005, the U.S. Department of Education awarded eight Striving Readers grants to projects around the country. The goal of Striving Readers is to improve literacy skills and achievement for struggling readers in middle and high school and to increase the research base using randomized clinical research to address improvement in adolescent…

An assessment framework is like a blueprint, laying out the basic design of the assessment by describing the mathematics content that should be tested and the types of assessment questions that should be included. It also describes how the various design factors should be balanced across the assessment. This is an assessment framework, not a…

A popular design in large-scale educational assessments as well as any other type of survey is the balanced incomplete block design. The design is based on an item pool split into a set of blocks of items that are assigned to sets of "assessment booklets." This article shows how the problem of calculating an optimal balanced incomplete block…

Developing the Trends in International Mathematics and Science Study (TIMSS) 2007 Assessment Frameworks represents an extensive collaborative effort involving individuals and expert groups from more than 60 countries around the world. The document contains three frameworks for implementing TIMSS 2007--the Mathematics Framework, the Science…