The purpose of this study is to extend the work of Spybrook and Raudenbush (2009) and examine how the research designs and sample sizes changed from the planning phase to the implementation phase in the first wave of studies funded by IES. The authors examine the impact of the changes in terms of the changes in the precision of the study from the…

The General Certificate of Education (GCE) A* grade was first awarded to students in 2010. It was introduced to assist higher education institutions in differentiating between the highest performing students and to promote and reward greater stretch and challenge. This paper, based on a synthesis of key policy documents, an analysis of…

I define a treatment effect in terms of a comparison of outcomes and provide a typology of all possible comparisons that can be used to estimate treatment effects, including comparisons that are relatively unknown in both the literature and practice. I then assess the relative merit, worth, and value of all possible comparisons based on the…

Designing and implementing assessment instruments are ongoing challenges for physical educators. The initial design phase includes asking the question, "What do I want the students to know and/or be able to do?" Seemingly, the question is direct and should lead to a concrete answer. In addition to designing assessment instruments, one of the most…

A large body of literature documents the central importance of fidelity of program implementation in creating an internally valid research design and considering such fidelity in judgments of research quality. The What Works Clearinghouse (WWC) provides web-based summary ratings of educational innovations and is the only rating group that is…

Environment and sustainable development show how policies are becoming ever more complex and ambiguous. This trend calls for new evaluation approaches. They need to be more clearly focused on specific, explicit concerns. They must be driven by a strategic concept of use to overcome the vulnerability to manipulation of many integrative, essentially…

The purposes of metaevaluation go beyond the traditional functions of accountability and enhancement. It helps guide strategic organizational change and legitimizes evaluation systems. Metaevaluation results can also be used to create checklists so that the persons responsible for any evaluation can revise, monitor, and control them by themselves.…

In order to assess how completely a Small Learning Community (SLC) and/or Career Academy is implemented in any given site, and to connect the degree of implementation with the amount of improvement in student performance, two kinds of information are needed. The first is information on the quality of implementation, which can be collected by…

In consultation with experts in the fields of education research and whole school improvement, researchers at the Comprehensive School Reform Quality (CSRQ) Center created a framework for evaluating the scientific rigor of research studies that report on the efficacy of whole school improvement models. In this paper, the authors begin with a brief…

There is increasing recognition of the need for evaluations that identify program processes or mediators and assess degree of program implementation rather than focusing solely on outcome evaluation. This paper describes the application of complementary qualitative and quantitative evaluation procedures to assess the degree of implementation of…