This paper seeks to raise awareness among educational researchers and practitioners of some significant weaknesses and internal contradictions of randomised control trials (RCTs). Although critiques throughout the years from education scholars have pointed to the detrimental effects of this experimental approach on education practice and values,…

Many teacher education researchers have expressed concerns with the lack of rigorous impact evaluations of teacher preparation practices. I summarize these various concerns as they relate to issues of internal validity, external validity, and measurement. I then assess the prevalence of these issues by reviewing 166 impact evaluations of teacher…

This brief provides tips for forming a team of staff and consultants with the needed expertise to make key measurement decisions that will ensure high-quality data for answering the study's research questions. The brief outlines the main responsibilities of measurement team members. It also describes typical measurement tasks and discusses…

This guide is addressed to policy officials, program providers, and researchers who are seeking to: (1) identify and implement social programs backed by valid evidence of effectiveness; or (2) sponsor or conduct an evaluation to determine whether a program is effective. The guide provides a brief overview of which studies can produce valid…

Background: Youth programs often rely on self-reported data without clear evidence as to the accuracy of these reports. Although the validity of self-reporting has been confirmed among some high school and college age students, one area that is absent from extant literature is a serious investigation among younger children. Moreover, there is…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Quality measures were originally developed for research aimed at describing the settings that children spend time in and identifying the characteristics of these environments that contribute to children's development. They were also developed to guide improvements in practice. Increasingly, however, measures of quality are being used for further…

A syllabus analysis instrument was developed to assist program evaluators, administrators and faculty in the identification of skills that students use as they complete their college coursework. While this instrument can be tailored for use with a variety of learning domains, we used it to assess students' use of and exposure to computer…

Empirical literature on two types of hearings, and their applicability in evaluating educational programs, is reviewed. An introduction describes the use and nature of hearings and distinguishes adversary hearings, in which two parties argue opposing sides of an issue, from committee hearings, in which a variety of positions are presented to a…

Community coalitions (CCs) have labored with some difficulty to demonstrate empirical evidence of effectiveness in preventing a wide range of adolescent problem behaviors. Training and technical assistance (TA) have been identified as important elements in promoting improved functioning of CCs. A reliable, valid, and inexpensive method to assess…

Two separate sets of minimum standards designed to guide the evaluation of bilingual projects are proposed. The first set relates to the process in which the evaluation activities are conducted. They include: validity of assessment procedures, validity and reliability of evaluation instruments, representativeness of findings, use of procedures for…

This paper illustrates the need for full description and measurement of differences between "experimental" and "control" situations in school program evaluation studies. Two studies were completed at the University of Oregon involving the innovative program--differentiated staffing (DS). The first study, a doctoral investigation, sought to assess…

Thirty studies on supported employment for people with disabilities were evaluated using a proposed Social Validity Matrix. Results suggested further research should investigate alternative strategies for assessing social validity, develop systematic procedures for collecting and using social validity data, and establish functional variables…

The purpose of this study was to develop an instrument, the "Job Seeking Self-Efficacy Scale," to measure perceived self-efficacy in job-seeking activities. The construction of the instrument, which incorporated tasks that have been determined to be important in job seeking for individuals with disabilities, was based on Bandura's theory of…

This study describes validation data for qualitative and quantitative writing measures useful with a full range of students, including special education students, in grades three through five. The study looked for consistent administration and scoring, sensitivity in differentiating groups of students, relation to other accepted assessments, and…