The Council for the Accreditation of Educator Preparation (CAEP) requires faculty from educator preparation programs to provide evidence of credential candidates' impact on K-12 student learning. However, there is a paucity of information on preparation programs' use of direct assessments of student learning to gauge credential candidate…

Good evaluation requires that evaluation efforts themselves be evaluated. Many things can and often do go wrong in evaluation work. Accordingly, it is necessary to check evaluations for problems such as bias, technical error, administrative difficulties, and misuse. Such checks are needed both to improve ongoing evaluation activities and to assess…

A large body of literature documents the central importance of fidelity of program implementation in creating an internally valid research design and considering such fidelity in judgments of research quality. The What Works Clearinghouse (WWC) provides web-based summary ratings of educational innovations and is the only rating group that is…

Many consider online courses to be an inferior alternative to traditional face-to-face (f2f) courses because exam cheating is thought to occur more often in online courses. This study examines how the assessment design in online courses contributes to this perception. Following a literature review, the assessment design in a sample of online…

This article describes the design and evaluation approaches to address the complexity posed by systems change initiatives. The role of evaluations in addressing the following issues is briefly reviewed: moving from strategic planning to implementation, impacts on system-level coordination, anticipated timeline of impact, and individual level…

Drug studies are often cited as the best exemplars of evaluation design. However, many of these studies are seriously biased in favor of positive findings for the drugs evaluated, even to the point where dangerous effects are hidden. In spite of using randomized designs and double blinding, drug companies have found ways of producing the results…

This paper describes the process of designing and administering a sufficiently valid and reliable assessment instrument to measure the progress in attainment of adult learners studying numeracy, and discusses some of the inherent difficulties that were involved. The fieldwork took place during 2003-2005 and involved a sample of 34 teachers and 412…

The gold standard in making causal inference on program effects is a randomized trial. Most randomization designs in education randomize classrooms or schools rather than individual students. Such "clustered randomization" designs have one principal drawback: They tend to have limited statistical power or precision. This study aims to…

This is a critical review of methodological issues in the evaluation of adult literacy education programs in the United States. It addresses the key research questions: What are the appropriate methods for evaluating these programs under given circumstances. It identifies 15 evaluation studies that are representative of a range of adult literacy…

The purpose of this article is to extend the discussion of issues currently being debated on the need for more rigorous program evaluation in educational and other sectors of research, to the field of international development evaluation, reviewing the different approaches which can be adopted to rigorous evaluation methodology and their…

Course evaluation is integral to medical education. We discuss (1) distinctive features of medical education that impact on course evaluation, (2) a framework for course evaluations, (3) details that shape the evaluation process, (4) key measurement issues important to data gathering and interpretation, and (5) opportunities for expanding the…

Discussed is the issue of the appropriateness of applied science approaches to evaluation in environmental education. The relationships between characteristics of applied science approaches to evaluation and the special characteristics of environmental education are explored. (Author/YP)

In October 2002, the Institute of Education Sciences (IES) contracted with Mathematica Policy Research, Inc. (MRP) to help identify issues pertinent to the evaluation of Title I and to propose feasible evaluation design strategies. This design effort took its lead from two sources: (1) the Title I Independent Review Panel (IRP); and (2) a more…

This volume brings together the reports and discussion connected with a roundtable convened by the Departments of Education and Labor and the National School-to-Work Office to discuss issues surrounding the conduct of a net impact evaluation of school-to-work. After an introduction that summarizes the approach and intentions of the School-to-Work…

This article describes the use of a schematic model for developing and distributing online surveys. Two empirical studies that developed and implemented online surveys to collect data to measure satisfaction in various aspects of human resource development and adult education exemplify the use of the model to conduct online survey research. The…