Engaging students in creating learning resources has demonstrated pedagogical benefits. However, to effectively utilize a repository of student-generated content (SGC), a selection process is needed to separate high- from low-quality resources as some of the resources created by students can be ineffective, inappropriate, or incorrect. A common…

Comparative judgment is gaining popularity as an assessment tool, including for high-stakes testing purposes, despite relatively little research on the use of the technique. Advocates claim two main rationales for its use: that comparative judgment is valid because humans are better at comparative than absolute judgment, and because it distils the…

The George Mason University (Mason) offices responsible for institutional reporting and assessment were merged to strengthen and solidify their functions and expertise. The organizational structure creating the Office of Institutional Effectiveness and Planning (OIEP) was intended to engineer a new paradigm where assessment would inform the…

Scoring procedures for the constructed-response (CR) items in large-scale mixed-format educational assessments often involve checks for rater agreement or rater reliability. Although these analyses are important, researchers have documented rater effects that persist despite rater training and that are not always detected in rater agreement and…

Science, Technology, Engineering, and Mathematics (STEM) education has become increasingly important in the US due to its influence on the nation's educational needs, the creation of a skilled labor force, and opportunities for more tech-savvy workers. However, the evaluation approaches and methodologies used in STEM education programs have come…

The act of valuing in an evaluation may be perceived in different ways. We consider the multiple theoretic perspectives that govern an evaluator's behavior and present a typology of evaluator valuing roles. Within this typology we describe three ways in which value judgments are typically reached--by stakeholders alone, stakeholders and evaluators…

Despite the present economic difficulties facing all levels of the nation's educational enterprise, a significant portion of education's resources continue to be used for evaluative activities. As a result, presumably competent evaluation personnel are sought after to evaluate activities in nearly every area of education and its cognate fields.…

A sample of educational evaluators was asked to design an outcomes-focused evaluation of a school program that aims to improve the academic achievement and self-esteem of students. Evaluators provided detailed descriptions of their evaluation design and methodology in their responses. These descriptions were coded and analyzed to determine the…

Competency-based assessment and an emphasis on obtaining higher-level outcomes that reflect physicians' ability to demonstrate their skills has created a need for more advanced assessment practices. Simulation-based assessments provide medical education planners with tools to better evaluate the 6 Accreditation Council for Graduate Medical…

This excerpt from the opening plenary asks evaluators to consider two questions regarding learning and evaluation: (a) How do evaluators know if, how, when, and what people are learning during an evaluation? and (b) In what ways can evaluation be a learning experience? To answer the first question, evaluators can apply the commonplaces of…

The Office of Planning, Research and Evaluation (OPRE), a unit within the Administration for Children and Families (ACF), is responsible for advising the Assistant Secretary for Children and Families on increasing the effectiveness and efficiency of programs to improve the economic and social well-being of children and families. In collaboration…

The author presents his opinion that educational evaluation has much similarity to the nonfiction narrative, (defined as a series of events ordered in time), particularly as it relates a current situation to future possibilities. He refers to Stake's statement that evaluation is concerned not only with outcomes but also with antecedents and with…

Values and beliefs underlying assumptions concerning the independent evaluations of persons and programs involved in the educational process are discussed. Particular emphasis is placed on M. Scriven's treatise on bias in evaluation (1976). Works by E. R. House, R. E. Stake, L. Stenhouse, M. Parlett and D. Hamilton, and D. L. Stufflebeam are also…

Addresses the role of evaluation, the basic logic of evaluation, and how the field of evaluation is structured. Focuses on evaluation in applied psychology, and describes the six conventional fields of evaluation. (SLD)

The dimensions of educational evaluation include conceptualization and technology of evaluation. The methods have been drawn from the area of educational and psychological research. The techniques of evaluation need to serve the information needs of clients, address the central value issues, deal with situational realities, meet requirements of…