This "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)" provides a detailed description of the standards and procedures of the What Works Clearinghouse (WWC). The remaining chapters of this Handbook are organized to take the reader through the basic steps that the WWC uses to develop a review protocol, identify…

Statistical significance is not a sufficient condition for claiming a hypothesis has been supported. Constructive replications are more important. Statistically significant results may be meaningless while a sequence of nonsignificant results may be quite important. Gives advice on how to overcome some limitations of classifical statistical…

The basic notion of variability is generalized from a univariate context to a multivariate context using two matrix functions, a determinant, and a trace, yielding a number of alternative multivariate indices of shared variation. Some problems in the interpretation of tests of multivariate hypotheses are reviewed. (Author/BW)

When analysis of variance is used, statistically significant differences may or may not be of practical significance to educators. A large part of the problem is due to the fact that a "zero difference" null hypothesis can always be rejected statistically if the sample size is large enough. If, however, a method based on the noncentral F…

Shares three ideas concerning how to evaluate the practical importance of developmental findings to make them more useful to policymakers: (1) statistical significance tests need to be accompanied by effect size estimates; (2) meta-analyses are helping in using all existing data when examining issues involved in policy debates; and (3) researchers…

Theories employed to explain regularities in social behavior often contain explicit or implicit reference to the presence of nonlinear and/or nonadditive (i.e., multiplicative) relationships among germane variables. While such nonadditive features are theoretically important, the inclusion of quadratic or multiplicative terms in structural…

An investigation was conducted of the characteristics of two estimation procedures and corresponding test statistics used in the analysis of completely randomized factorial experiments when observations are lost at random. For one estimator, contrast coefficients for cell means did not involve the cell frequencies. For the other, contrast…

A reanalysis of research reported in the "Journal of Industrial Teacher Education" demonstrates a tendency among researchers to base interpretations of data entirely on the results of tests for statistically significant differences. The appraisal suggests that several of the published findings should be reviewed using additional data…

Presentation of quantitative research on the family often suffers from a tendency to interpret findings on a statistical rather than substantive basis. Advocates the use of data analysis that lends itself to an intuitive understanding of the nature of the findings, the strength of the association, and the import of the result. (JPS)

Abuses and misuses of statistics are frequent. This digest attempts to warn against these in three broad classes of pitfalls: sources of bias, errors of methodology, and misinterpretation of results. Sources of bias are conditions or circumstances that affect the external validity of statistical results. In order for a researcher to make…

The general goal of this paper is to help researchers conduct appropriately designed goodness of fit studies for item response model applications. The specific purposes are to describe: (1) an up-to-date set of promising and useful methods for addressing a variety of goodness of fit questions; and (2) current research studies to advance this set…

Too few researchers understand what statistical significance testing does and does not do, and consequently their results are misinterpreted. This Digest explains the concept of statistical significance testing and discusses the meaning of probabilities, the concept of statistical significance, arguments against significance testing,…

When two groups, initially dissimilar, undergo different treatments, can subsequent differences be partitioned in such a way that the difference between the two treatments is unbiased? This is the central problem of this paper, and it is confronted by the examination of two levels of information using a Follow Through Evaluation. The first…

The uses of data on siblings for educational and psychological research purposes are explained. Examples are based on empirical data. Five types of research problems, using sibling data, are explored: (1) statistical control of family background variance in psychological or educational studies which use analysis of covariance (ANCOVA) or multiple…