Designs for sequential sampling procedures that adapt to cumulative information are discussed. A familiar illustration is the play-the-winner rule in which there are two treatments; after a random start, the same treatment is continued as long as each successive subject registers a success. When a failure occurs, the other treatment is used until…

A statistical analysis procedure is developed, based on the notion that many educational programs are dynamic interventions in natural growth processes, and is called value-added analysis. The theory of value-added analysis, and several applications are presented. (Author/JKS)

The authors demonstrate how exponential smoothing can play a role in the identification of the intervention component of an interrupted time-series design model that is analogous to the role that the sample autocorrelation and partial autocorrelation functions serve in the identification of the noise portion of such a model. (Author/BW)

A method is presented for analyzing data collected in a multiple group time-series design. This consists of testing linear hypotheses about the experimental and control group-means. Both a multivariate and a univariate procedure are described. (Author/GK)

The equating methods and sampling designs used in the empirical studies in this special issue on the use of matched samples for test equating are described. Four requisites for equating are listed, and the invariance of equating functions requisite is identified as the focus of this issue. (SLD)

The concepts, definitions, and notation that have evolved with the development of telephone survey design methodology are discussed and presented as a unified structure. This structure is then applied to some of the more well-known telephone survey designs and alternative designs are developed. The relative merits of the different survey designs…

Various methods have been suggested for the analysis of data collected in research settings where random assignment of subjects to groups has not occurred. For the purposes of this paper the set of allowable nonrandomized designs is made up of those research designs where data are collected for one or more groups of subjects at two or more time…

This final report of a National Institute of Education project explores Bayesian statistical analysis as a paradigm for educational impact studies, particularly studies on the education of the disadvantaged. The position of the report is that much of what is wrong with educational research can be attributed to the use of an inappropriate model for…

The effects of using different data analysis methods on estimates of treatment effects of educational programs were investigated. Various regression models, such as those recommended for Title I program evaluations, were studied. The first effect studied was the amount of bias that might be expected to occur in the various settings. Results…

Model identification of time-series data is essential to valid statistical tests of intervention effects. Model identification is, at best, inexact in the social and behavioral sciences where one is often confronted with small numbers of observations. These problems are discussed, and the results of independent identifications of 130 social and…

The validity of the equipercentile hypothesis of the Title I Evaluation and Reporting System norm-referenced evaluation model was examined using 3,224 seventh- and ninth-grade students. Findings from confidence interval procedures contradicted the equipercentile hypothesis. There was a pattern of large gains for students not receiving any special…

The effects of certain characteristics (e.g., sample design) of National Assessment of Educational Progress (NAEP) data on statistical analysis techniques are considered. Ignoring special features of NAEP data and proceeding with a standard analysis can produce inferences that underestimate the true variability and overestimate the true degrees of…

Six models for evaluating change are examined via a Monte Carlo study. All six models show a lack of power. A modified analysis of variance procedure is suggested as an alternative. (JKS)

Statistical methods for program evaluation with nonexperimental data are reviewed with emphasis on circumstances in which nonexperimental data are valid. Three solutions are proposed for problems of selection bias, and implications for evaluation design and data collection and analysis are discussed. (SLD)

The application of time series analysis is described, focusing on the use of regression analysis for analyzing time series in a way that may make it more readily available to an evaluation practice audience. Practical guidelines are suggested for decision makers in government, health, and social welfare agencies. (SLD)