A method called multi-attribute utility analysis (MAUA) provides a decision-making framework that facilitates comparative analysis of multiple real-world decision alternatives with unique complex attributes. Utility analysis as a measure of effectiveness has been minimally used by educational researchers to date, despite clear relevance in complex…

Suggests an approach to providing practical evaluation training through university-supported, long-term funded evaluation projects. Such projects are managed through a university center that provides assistance to clients in student assessment and program evaluation. Discusses the benefits and challenges of these experiences for students. (SLD)

Reviews the literature on the evaluation of instructional technology and describes the evaluation of a campus-wide learning technology effort at the University of Illinois at Urbana-Champaign. This evaluation used multiple means over 3 years, including surveys of 7,140 students. Discusses insights from this project for future efforts to evaluate…

Describes the conceptual scheme for a 1-year evaluation course designed to each the competencies required to conduct evaluations that provide the sense-making for informed decision making. Draws on the authors' experience in developing such a course over 4 years. (SLD)

Focuses on the ideal future of evaluation as a discipline, including evaluation in the nonacademic world and evaluation as it applies to elementary, secondary, and higher education. In the ideal future, evaluation will be fully accepted as a key discipline and will be fully used in rational discourse and policy decisions. (SLD)

Describes the Northwestern training program in methodology and evaluation research, discussing its faculty, curriculum, research opportunities, and emotional climate. (SLD)

This paper provides the results of a literature review on the use of practical, hands-on training experiences in evaluation course work and training programs. The review spans the years 1965-2003. I identified 18 articles that encompass four basic approaches for practical evaluation training: simulation, role-play, single course projects, and…

This article describes the Self-Assessment of Program Evaluation Expertise instrument and procedure developed to help participants assess their learning gains in a 3-week evaluation institute. Participants completed the instrument in a pre- and posttest format. To reduce both the threat of embarrassment from individual results and the temptation…