Systematic reviews have witnessed significant growth across many fields, including education. In this article, we outline the background of this growth, highlight the tendency to focus on methodological considerations, and propose a framework to support education researchers in preparing systematic reviews with broad impact. We draw on our…

Research syntheses in education, particularly meta-analyses and best-evidence syntheses, identify evidence-based practices by combining findings across studies whose constructs are similar enough to warrant comparison. Yet constructs come preloaded with social, historical, political, and cultural assumptions that anticipate how research problems…

Research syntheses are appealing because they enable decision makers to determine quickly whether policies, programs, and practices will have effects on student achievement and, if so, the magnitudes of the likely effects. Such syntheses should present objective, clear, scientifically accurate, and defensible evidence in terms that educators can…

Syntheses of research on educational programs have taken on increasing policy importance. Procedures for performing such syntheses must therefore produce reliable, unbiased, and meaningful information on the strength of evidence behind each program. Because evaluations of any given program are few in number, syntheses of program evaluations must…

Traditional methods for preparing systematic reviews and syntheses of effectiveness studies rely on a limited set of methodological criteria to include studies that measure and report effects too narrowly to forward the mission of evidence-based practice. This article discusses why and how the criteria for study selection, evidence screening, and…

When causal inferences are to be synthesized across multiple studies, efforts to establish the magnitude of a causal effect should be balanced by an effort to evaluate the generalizability of the effect. The evaluation of generalizability depends on two factors that are given little attention in current syntheses: construct validity and external…

This response to Slavin's article (2008) explores the issues of transparency, representation, and warranting of claims in Slavin's descriptions of the work of others and his suggestions for program evaluation syntheses. Through contrastive analyses between Slavin's representations of the program evaluation synthesis efforts of five organizations…

As scholars and their audiences pursue standards of evidence, standards for literature reviews have also become salient. Many authors advocate "systematic" reviews and articulate standards for these. This article compares the bodies of literature derived from systematic and other types of review, which the author labels conceptual, and…

In comparing several research and evaluation studies, the author contends that variation in outcomes may provide clues to valuable information. Five examples are used to illustrate how conflicting findings can be useful. (RLV)

Describes a study of reliability among coders of information for meta-analysis (a quantitative procedure for synthesizing data in primary research reports) of research on life satisfaction in American adults. Identifies sources and areas of disagreement among coders and discusses measures that can be used to enhance intercoder consistency.…

Most of the variance in educational effectiveness studies is inexplicable in terms of influences that can be measured and controlled. Nevertheless, it is still possible to design educational policy that will function well under conditions of uncertainty. (Author/RLV)

Reviews "Soviet Studies in Mathematics Education: Volume 2. Types of Generalization in Instruction" (V. V. Davydov, National Council of Teachers of Mathematics, 1990), which is useful for educational researchers because its major theory of dialectical materialism offers a view of knowledge with a significant role for activities that…