Treatment effect estimates from a "regression discontinuity design" (RDD) have high internal validity. However, the arguments that support the design apply to a subpopulation that is narrower and usually different from the population of substantive interest in evaluation research. The disconnect between RDD population and the…

This study uses a multi-level multivariate propensity score matching approach to examine the synthetic cohort design (SCD) in estimating the schooling effect on mathematics proficiency of the focal cohort 2 (8th graders). Collecting 7th and 8th graders at the same time point, the SCD is sufficient in estimating the schooling effect under the…

Describes the use of multilevel models (MLMs) for studies in which individuals are randomized by groups to treatment condition. Uses data from the Safer Choices study, an evaluation of a theory-based multi-component program to prevent sexually transmitted diseases and pregnancy, to illustrate the application of MLMs for both continuous and…

The Comparing Options in Secondary Mathematics: Investigating Curriculum (COSMIC) project is a longitudinal study of student learning from two types of mathematics curricula: integrated and subject-specific. Previous large-scale research studies such as the National Assessment of Educational Progress (NAEP) indicate that numerous variables are…

One source of bias, regression artifacts, in program evaluations that use regression discontinuity design is identified, and its importance is discussed. A method is suggested for its correction. Although use of this method does not solve the problem of selection bias, it does improve the reliability of nonexperimental methods. (SLD)

Selectivity bias arises in program evaluation when the treatment or control status of the subjects is related to unmeasured characteristics that themselves are related to the program outcome under study. This situation has the potential to lead to an incorrect estimation of the treatment effect when assignment to treatment and control groups is…

Statistical procedures of the regression discontinuity and regression projection designs were compared to determine which design was more able to detect treatment effect in remedial or gifted programs. Neither test was more precise in all situations. Tables list the more precise test usder specific conditions. The regression projection model…

Focuses on the fact that an educational treatment typically involves an intervention in a growth process. By modelling this process, expected growth for various treatment groups under control conditions may be estimated. Actual growth can be compared with projected growth to estimate the value-added by the program. A simple model is developed. (RC)

A solution to the problem of specification error due to excluded variables in statistical models of treatment effects in nonrandomized (nonequivalent) control group designs is presented. It involves longitudinal observation with at least two pretests. A maximum likelihood estimation program such as LISREL may provide reasonable estimates of…

The RMC Research Corporation evaluation model C1--the special regression model (SRM)--was evaluated through a series of computer simulations and compared with an alternative model, the norm referenced model (NRM). Using local data and national norm data to determine reasonable values for sample size and pretest posttest correlation parameters, the…

The accuracy with which regression models estimate treatment effects is dependent upon a number of conditions. The stability of the regression line (a function of sample size and correlation between pretest and posttest) is said to be the most important of these conditions. The utility of regression models is proportional to the size of the…

Structural equation models reanalyzed data from the 1969 Westinghouse Head Start evaluation and extended the analysis to blacks and full-year programs. Results showed that the selection bias for summer program whites was absent for blacks and full-year program whites, and that Head Start helped the most disadvantaged children. (SLD)

This report documents the activities of the technical committee formed to examine issues related to the norm-referenced model (RMC Model A) for evaluation of Elementary and Secondary Education Act Title I programs. Background information is provided on the committee's activities, purpose, and meetings; and the members are listed. The reports of…

This study explored the assumption that the percentile rank of a group of students should be expected to remain the same on a post-test as on a pretest when the treatment of the group in the intervening period was the same as the treatment of the norms population. This assumption is basic to the use of the norm-referenced model for the assessment…

The norm-referenced evaluation model (RMC Model A) for Title I project evaluation, consists of procedures whereby the expected posttest standing of a treatment group under the null condition is generated from their pretest standing. It is assumed that the treatment group is not selected on the basis of their pretest scores and can be considered…