In this response, we first show that Simpson's proposed analysis answers a different and less interesting question than ours. We then justify the choice of prior for our Bayes factors calculations, but we also demonstrate that the substantive conclusions of our article are not substantially affected by varying this choice.

A recent paper uses Bayes factors to argue a large minority of rigorous, large-scale education RCTs are "uninformative." The definition of "uninformative" depends on the authors' hypothesis choices for calculating Bayes factors. These arguably overadjust for effect size inflation and involve a fixed prior distribution,…

Comments on S. Pogrow's suggestion that educational programs should be judged by the degree to which schools using them achieve "surprising scores." Emphasizes the importance of the use of control groups in program evaluation. (SLD)

Federal support for answers to the question "What is really going on out there?" has increased interest in the use of ethnography as a research method for evaluation of educational programs. Pros and cons of the use of ethnography for educational research are discussed. (PR)

Explores issues in evaluating exemplary programs, including the measurement of the degree to which the program is exemplary or promising. Suggests that comparison (control) group analysis can mislead researchers about the effectiveness of a program and suggests the use of gain scores to identify exemplary programs. (SLD)

Although important progress has been made in educational evaluation during the past decade new problems believed to be more serious threaten the progress that has been made. These problems are said to be a mixture of logistics and politics and are in large part an outgrowth of the increasing pluralism of American society. (Author/AM)

Answers criticisms by Mary Anne Bunda, Ernest House, and Mary Kennedy. Discusses, among other issues, legal authorization for the use of higher quality evaluation designs and the use of evaluation research results. (GC)

Education researchers, practitioners, and policymakers alike are committed to identifying interventions that teach students more effectively. Increased emphasis on evaluation and accountability has increased desire for sound evaluations of these interventions; and at the same time, school-level data have become increasingly available. This article…

Six deficits of this model are discussed: disparity in proponent prowess, fallible arbiters, excessive confidence in the model's potency, difficulties in framing the proposition in a manner amenable to adversary resolution, "a cat's-paw for biased decision makers," and, excessive costs. (Author/JM)

Presents a case study of how the review and clearance procedures are operating for evaluation instruments used in collecting information in connection with studies for the U.S. Office of Education. The review procedures are defective: They consume too much time. They are very costly. The results are unproductive. (Author/JM)

Two issues in the Holtzman Project bear examination and reconsideration: (1) the recommendation for the use of field experiments as an exclusive "authorized" evaluation design, and (2) the focus on the needs of the federal client, and the apparent lack of concern for clients at the state and local levels. (GC)

Summarizes a report from Northwestern University that reviewed Federal evaluation practices of federally supported educational programs at the national, state, and local levels. Considers: (1) evaluation purposes and methods; (2) evaluators' capabilities; (3) use of evaluation results; and (4) ways that evaluation procedures/practices can be…

Responds to a critique of this author's article on consideration of purpose and intended use when making evaluations of assessments, commenting on three areas from that critique that warrant further discussion: considering the uses of assessment tools; global quality versus content specificity; and creating a toolkit of measures for examining…

Discusses the following aspects of educational evaluation: (1) structural changes; (2) conceptual changes; (3) mixed methods and unraveling consensus; (4) utilization of findings; (5) the role of values; and (6) the role of politics. Discusses how evaluation moved from monolithic to pluralistic, reflecting the change from consensus to pluralism in…

Notes specific concerns with Dickinson's (2002) assertion that the processes used to measure the quality of out-of-home care for young children have begun to lag behind recent developments in the field and that there is a lag in the National Association for the Education of Young Children's accreditation standards. Questions the usefulness of…