Many teacher education researchers have expressed concerns with the lack of rigorous impact evaluations of teacher preparation practices. I summarize these various concerns as they relate to issues of internal validity, external validity, and measurement. I then assess the prevalence of these issues by reviewing 166 impact evaluations of teacher…

This guide is addressed to policy officials, program providers, and researchers who are seeking to: (1) identify and implement social programs backed by valid evidence of effectiveness; or (2) sponsor or conduct an evaluation to determine whether a program is effective. The guide provides a brief overview of which studies can produce valid…

While there has been some increasing interest in external validity, most work to this point has been in assessing the similarity of a randomized trial sample and a population of interest (e.g., Stuart et al., 2010; Tipton, 2011). The goal of this research is to calculate empirical estimates of the external validity bias in educational intervention…

This issue of "FOCUS" discusses external validity and what rehabilitation researchers can do to help practitioners answer the question "How far can we generalize this finding-- is it applicable to other clients/patients, with different characteristics, in dissimilar settings treated by other clinicians?," which clinicians and…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Quality measures were originally developed for research aimed at describing the settings that children spend time in and identifying the characteristics of these environments that contribute to children's development. They were also developed to guide improvements in practice. Increasingly, however, measures of quality are being used for further…

This study attempted to validate the use of academic growth and development items from Tennessee alumni surveys as measures of program quality and effectiveness at the University of Tennessee (UTK), Knoxville. The argument is made that it is essential that the instruments used to assess students educational outcomes be valid measures of the goals…

This review evaluates research on the treatment of challenging behavior in school settings for students with autism spectrum disorders (ASD). Electronic database searches were carried out to identify studies published between 1995 and 2005. Twenty-six studies were identified. A variety of procedures were implemented in these studies to decrease…

Community coalitions (CCs) have labored with some difficulty to demonstrate empirical evidence of effectiveness in preventing a wide range of adolescent problem behaviors. Training and technical assistance (TA) have been identified as important elements in promoting improved functioning of CCs. A reliable, valid, and inexpensive method to assess…

A study of 24 videotaped interviews with sexually abused children (ages 4-12) investigated the effectiveness of using anatomically correct dolls during the interview process. Results found that when children were interviewed without dolls they gave longer and more detailed responses, suggesting that use of the dolls inhibited rather than…

This study sought to assess the social validity of teacher and parent implemented assessment-based intervention for challenging behaviour in young children. The Behavior Intervention Rating Scale was completed by adults at the end of intervention for each participant. Overall ratings were high for both intervention acceptability and intervention…

Recent approaches to empirical derivation and validation of alcoholic subtypes which involve cluster analysis wihin a single multivariate assessment domain have been questioned. To investigate more clinically meaningful subtypes in a sample of 245 male alcoholics, cluster analysis was performed on a combination of variables from four assessment…

This paper discusses the practice of social validity assessment in applied behavior analysis, proposes elements that should be included in state-of-the art assessment procedures, offers a taxonomy of consumers to guide the construction and circulation of assessments, and suggests strategies to improve social validity assessment. (JDD)

This paper argues that many social validity processes in applied behavior analysis are actually measuring consumer satisfaction and not the habilitative validity of goals, procedures, or outcomes. The term "habilitative validation" is proposed to replace social validity, and use of more objective assessment methods is encouraged.…

A study evaluated the effectiveness of pretesting infantry soldiers scheduled to take written Skill Qualification Tests (SQT) in order to determine their training needs prior to official testing. During the study, researchers developed and field-tested a pretest for the SQT that was based on the 11B1 Infantryman Soldier's Manual and the 1980 SQT…