Purpose: The purpose of this paper is to focus on challenges faced by longitudinal quantitative analyses of school improvement processes and offers a systematic literature review of current papers that use longitudinal analyses. In this context, the authors assessed designs and methods that are used to analyze the relation between school…

Scientifically based research used to inform evidence based school reform efforts has been required by the federal government in order to receive grant funding since the reenactment of No Child Left Behind (2002). Educational evaluators are thus faced with the challenge to use rigorous research designs to establish causal relationships. However,…

A previous Monte Carlo study examined the relative powers of several simple and more complex procedures for testing the significance of difference in mean rates of change in a controlled, longitudinal, treatment evaluation study. Results revealed that the relative powers depended on the correlation structure of the simulated repeated measurements.…

Investigated the consistency and loci of leniency, halo, and range restriction effects in performance ratings in a longitudinal study. Policy supervisors (N=90) rated 350 subordinates on five occasions. Concluded that reliable variance in mean ratings is partly attributable to ratees, but mainly introduced by raters. (JAC)

Factors that influence the sample size necessary for longitudinal evaluations include the nature of the evaluation questions, nature of available comparison groups, consistency of the treatment in different sites, effect size, attrition rate, significance level for statistical tests, and statistical power. (Author/GDC)

Designed to identify sources for planning longitudinal evaluation strategies, this paper provides an overview and bibliography of resources regarding vocational follow-up and presents a conceptual model for longitudinal program assessment. Introductory material highlights major works in the area of vocational program assessment and summarizes the…

The self-selection problem in program evaluation is explored, and two ways of addressing it, covariate control approaches and two-stage sample selection models, are described. A longitudinal study of the effects of participation in Alcoholics Anonymous illustrates both approaches and shows the advantages of the two-step procedure. (SLD)

This report summarizes tasks undertaken to assess the feasibility of a longitudinal analysis of elementary and secondary schools. The initial objective was to see if it was technically possible to analyze overlap samples of schools from the Schools and Staffing Survey (SASS) to produce substantive findings. Over the course of the project various…

Outcome research in counseling has generally overlooked the problem that main effects of treatment may have little meaning in the presence of interactions, and that variables not represented in research designs have no opportunity to demonstrate their interactive effect. Comparison of several groups in terms of mean differences or average gains,…

The evaluation of the preschool program conducted by the High/Scope Educational Research foundation is an example of a program evaluation that employed a randomized design and also examined long-term outcomes. The study, which employed a 22-year follow-up study, is important for its findings and its methodology. (SLD)

Strategies are presented for analyzing longitudinal research designs with many waves of data using hierarchical linear modeling. The approach defines well-focused parameters that yield meaningful effect size estimates and significance tests. It is illustrated with data from the Boys Town Follow-Up Study. (SLD)

Attrition, or the loss of study subjects, confounds the subject identification process. Design attrition occurs when subjects initially involved in the study are excluded because of changes in study focus or rationale. Data collection attrition occurs when subjects are excluded because they were not contacted by the data gatherer, or because the…

Articles in this special issue suggest improvements in longitudinal program evaluation to meet the challenges of evaluation of human services programs. Contributions describe innovations in design, methods, data analysis, utilization, and interpretation of the findings of longitudinal studies. (SLD)

A repeated measures design was used to assess the effects of a methadone maintenance program in several California locations. Subjects were 720 Chicano and Anglo men and women participating in rehabilitation programs for heroin addicts. The subjects were interviewed between 1978-1981, and the average follow-up time after initial treatment was 4-6…

This document reports the pilot test of the two components of the Child and Family Mental Health (CFMH) Evaluation Project -- the impact evaluation component and the in-depth evaluation component. (The impact evaluation is designed to determine the effects of the two primary prevention models of service and activities on the CFMH Head Start…