In the field of special education there is a dearth of group experimental studies that establish evidence-based practice. The effort to establish evidence-based practice has been associated with emphasizing experiments by using randomized controlled trial with large numbers of participants who are randomly assigned to a treatment. However,…

Large-scale randomized studies provide the best means of evaluating practical, replicable approaches to improving educational outcomes. This article discusses the advantages, problems, and pitfalls of these evaluations, focusing on alternative methods of randomization, recruitment, ensuring high-quality implementation, dealing with attrition, and…

It is currently unknown how many Institutional Review Boards (IRBs) exist at community colleges. This study investigates the percentage of public and Tribal community colleges that have active IRBs. It also examines the potential relationships between states that allow community colleges to confer baccalaureate degrees, or that have articulation…

To date, evaluation research and policy analysis have focused mainly on average program impacts and paid little systematic attention to their variation. Recently, the growing number of multi-site randomized trials that are being planned and conducted make it increasingly feasible to study "cross-site" variation in impacts. Important…

Spatial or temporal data mining tasks are performed in the context of the relevant space, defined by a spatial neighborhood, and the relevant time period, defined by a specific time interval. Furthermore, when mining large spatio-temporal datasets, interesting patterns typically emerge where the dataset is most dynamic. This dissertation is…

This NCEE Technical Methods report examines how to address the problem of missing data in the analysis of data in Randomized Controlled Trials (RCTs) of educational interventions, with a particular focus on the common educational situation in which groups of students such as entire classrooms or schools are randomized. Missing outcome data are a…

Discusses criticisms of null hypothesis significance testing (NHST), suggesting that historical use of NHST was reasonable, and current users should read Sir Ronald Fisher's applied work. Notes that modifications to NHST and interpretations of its outcomes might better suit the needs of modern science. Concludes that NHST is most often useful as…

To determine the effect of violating the assumption of homogeneity of covariance for the Tukey Wholly Significant Difference (WSD) test, Monte Carlo simulations varied the number of treatment groups, sample size, and degree of covariance heterogeneity. As covariance heterogeneity was increased, the empirical significance levels increased beyond…

The purposes of the present paper are to address the historical development of statistical significance testing and to briefly examine contemporary practices regarding such testing in the light of these historical origins. Precursors leading to the advent of statistical significance testing are examined as are more recent controversies surrounding…

Outlines concerns that must be addressed by those who advocate replacing statistical hypothesis-testing with alternative data-analysis strategies. Suggests that commonly recommended alternatives are not perfect and that various hypothesis-testing modifications can be implemented to make the process and its conclusions more credible. Hypothesis…

The results of an extensive analysis of the statistical methodology employed by researchers whose manuscripts were published in the first twelve volumes of the "Educational Administration Quarterly" are reported. (Author)

Statistical significance is not a sufficient condition for claiming a hypothesis has been supported. Constructive replications are more important. Statistically significant results may be meaningless while a sequence of nonsignificant results may be quite important. Gives advice on how to overcome some limitations of classifical statistical…

The interpretation of significant left-tailed analysis of variance (ANOVA) F-ratios is supported by considering the case of a fixed effects ANOVA model. The conclusions of this case are generalizable to other standard ANOVA models. (Author/PN)

Despite its widespread use in evaluation data analysis, statistical testing has come under persistent criticism resulting in calls for its rethinking, and even possible elimination (Carver, 1978, 1993). Saxon and Boylan issue a call "to strengthen developmental education research and to make it more accessible" (2003, p. 2). Among the types of…

Effect Sizes (ES) are an increasingly important index used to quantify the degree of practical significance of study results. This paper gives an introduction to the computation and interpretation of effect sizes from the perspective of the consumer of the research literature. The key points made are: (1) "ES" is a useful indicator of the…