This chapter describes a concrete process that stakeholders can use to make predictions about the future performance of programs in local contexts. Within the field of evaluation, the discussion of validity as it relates to outcome evaluation seems to be focused largely on questions of internal validity (Did it work?) with less emphasis on…

The National Assessment of Educational Progress (NAEP) program is a series of periodic assessments administered nationally to samples of students and designed to measure different content areas. This article describes a multi-year study that focused on the breadth of the development, administration, maintenance, and renewal of the assessments in…

The authors of this article consider how education researchers can improve school violence and school safety research by (a) examining gaps in theoretical, conceptual, and basic research on the phenomena of school violence; (b) reviewing key issues in the design and evaluation of evidence-based practices to prevent school violence; and (c)…

Assessments labeled as formative have been offered as a means to improve student achievement. But labels can be a powerful way to miscommunicate. For an assessment use to be appropriately labeled "formative," both empirical evidence and reasoned arguments must be offered to support the claim that improvements in student achievement can be linked…

In many school districts, the pressure to raise test scores has created overnight celebrity status for formative assessment. Its powers to raise student achievement have been touted, however, without attending to the research on which these claims were based. Sociocultural learning theory provides theoretical grounding for understanding how…

Provides an argument for incorporating consideration of consequences into validity theory that is grounded in the reflexive nature of social knowledge. It also calls for the consideration of evidence of validity based on the actual discourse surrounding the practices and products of testing. (SLD)

This paper proposes a model to establish which criteria are considered by stakeholders as valid for evaluating a program. The model is developed with the aim of increasing the credibility and use of evaluations. Stakeholders are involved in the identification of potential evaluation criteria, and ratings of validity and priority are used as the…

This article reviews experiences in the United States over the past 15 years in developing standards to guide and assess the work of professional evaluators. It introduces the Joint Committee on Standards for Educational Evaluation's "Program Evaluation Standards" and overviews the Committee's project to develop "Educational…

In program evaluation and policy analyses, utility supplants rigor as the first and most important criterion in judging literature reviews. Although R. T. Ogawa and B. Malen emphasize rigor, a utilization-focused evaluation emphasizes purpose and audience, both of which affect the conduct of a multivocal literature review. (SLD)

An approach for involving beneficiaries in specifying program attributes for evaluators to address is discussed. Methods are outlined for sharing influence among program decision makers and beneficiaries. Application of this approach in the evaluation of a problem-based medical school curriculum is described, with evidence on the enhancement of…

Seven chapters present 31 propositions challenging traditional ideas about the nature and practice of program evaluation. Methods to improve evaluation models and approaches and ways to address intermediate and advanced evaluation issues are explored. The discussion also serves as an introduction to the most analytical and comprehensive of the…

The Title I Evaluation and Reporting System (TIERS) was assessed to determine the effectiveness of the TIERS in meeting its statutory objectives. A second purpose was to outline directions for the future of the system in light of enactment of the Educational Consolidation and Improvement Act of 1981. Focus was on assisting the United States…

A system designed to assist training managers or evaluators in increasing the effectiveness of military training programs has been developed, tested, and refined. The Training Program Evaluation (TPE) system offers advantages over traditional approaches of assessing program effectiveness. TPE involves the direct observation of training and testing…

Two separate sets of minimum standards designed to guide the evaluation of bilingual projects are proposed. The first set relates to the process in which the evaluation activities are conducted. They include: validity of assessment procedures, validity and reliability of evaluation instruments, representativeness of findings, use of procedures for…

A study examined the information collected by the Department of Defense on both the quality of its new recruits and the effectiveness of its training in preparing recruits to operate in a technologically sophisticated environment. It found that data were collected at a recruit's entrance to military life, during and upon completion of formal…