This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Donald Campbell's approach to causal inference (D. T. Campbell, 1957; W. R. Shadish, T. D. Cook, & D. T. Campbell, 2002) is widely used in psychology and education, whereas Donald Rubin's causal model (P. W. Holland, 1986; D. B. Rubin, 1974, 2005) is widely used in economics, statistics, medicine, and public health. Campbell's approach focuses on…

This research provides a direct comparison of effect size estimates based on structural equation modeling (SEM), item response theory (IRT), and raw scores. Differences between the SEM, IRT, and raw score approaches are examined under a variety of data conditions (IRT models underlying the data, test lengths, magnitude of group differences, and…

In this study the authors investigated the use of 5 (i.e., Claudy, Ezekiel, Olkin-Pratt, Pratt, and Smith) R[squared] correction formulas with the Pearson r[squared]. The authors estimated adjustment bias and precision under 6 x 3 x 6 conditions (i.e., population [rho] values of 0.0, 0.1, 0.3, 0.5, 0.7, and 0.9; population shapes normal, skewness…

The purpose of this paper is to demonstrate how multiple linear regression provides a viable statistical methodology for dealing with meta-analysis in general, and specifically with the issues of nonindependence and design complexity, such as multiple treatments. Since the F-test and t-test are special cases of the general linear model,…

A general model is described which can be used to represent the four common types of meta-analysis: (1) estimation of effect size by combining study outcomes; (2) estimation of effect size by contrasting study outcomes; (3) estimation of statistical significance by combining study outcomes; and (4) estimation of statistical significance by…

Meta-analysis is a technique for combining the summary statistics from previously conducted research studies to indicate the direction of results and provide an index of the magnitude of effect size. This paper focuses on the effect of the violation of the assumption of independence (that the value of any included statistic is in no way…

Statistical methodologists have sometimes criticized the use of conventional statistics in meta-analysis, and in recent years a number of them have advocated the use of a special new statistical methodology for research synthesis. An examination of recent books describing this methodology shows that it is seriously limited in its applicability to…

This paper discusses distribution theory and power computations for four common "tests of combined significance." These tests are calculated using one-sided sample probabilities or p values from independent studies (or hypothesis tests), and provide an overall significance level for the series of results. Noncentral asymptotic sampling…

Two approaches for conceptualizing and estimating a contextual model for analyzing student achievement are the separate equation approach and the single equation approach. The separate equation method determines the relationship between the individual-level independent variable and the individual-level dependent variable within each group. It then…

Recent research casts doubt on the commonly held notions that administrators affect student learning through instructional leadership and that changing administrators will improve school performance. To help construct a model for examining the process of leader succession that specifies a number of major school process and outcome variables…

This study investigated seven methods for analyzing multivariate group differences. Bonferroni t statistics, multivariate analysis of variance (MANOVA) followed by analysis of variance (ANOVA), and five other methods were studied using Monte Carlo methods. Methods were compared with respect to (1) experimentwise error rate; (2) power; (3) number…

This study explored the efficacy of conjoint behavioral consultation (CBC) by assessing objective outcomes and social validity with a sample of students with and without diversity. Diversity characteristics that were investigated included ethnicity, socioeconomic status, family composition, maternal education level, and language spoken in the…

The purpose of this study was to determine the extent to which mastery testing contributes to instructional effectiveness and to determine the size of its effects in different outcome areas, such as aptitude and instructional time. A meta-analytic methodology was used to integrate findings from experimental studies. Twenty studies examined…

One of the most appropriate quasiexperimental approaches to compensatory education is the regression-discontinuity design. However, it remains underutilized, in part because of the need to clarify the link between the mathematical model and administrative decision-making. This paper explains the derivation of a program efficiency index congruent…