Given decision-makers often prioritize causal research that identifies the impact of treatments on the people they serve, a key question in education research is, "Does it work?". Today, however, researchers are paying increasing attention to successive questions that are equally important from a practical standpoint--not only does it…

Evaluators are often called upon to assess the effects of programs. To assess a program effect, evaluators need a clear understanding of how a program effect is defined. Arguably, the most widely used definition of a program effect is the counterfactual one. According to the counterfactual definition, a program effect is the difference between…

The "Program Administration Scale" (PAS) is designed to reliably measure and improve the leadership and management practices of center-based programs--the only instrument of its kind to focus exclusively on organization-wide administrative issues. In the third edition, the authors share updated information supporting the reliability and…

In 2021, the Council for Higher Education Accreditation (CHEA) made a monumental move to require postsecondary institutions to evaluate and document their actions to ensure fairness in admissions, an inclusive learning environment, and equitable student outcomes. Around the same time, a team comprising educational measurement experts, diversity…

In this practice brief, we offer recommendations for practitioners supporting low-income students, first-generation college students, students of color, and/or students who are academically underprepared. We focus specifically on practices related to students' psychosocial well-being. Our recommendations are based on a multiyear, mixed methods…

This document is written primarily for policy makers and state department of education staff who are considering through-year assessments, as well as consultants and contractors state departments rely on. The document identifies essential things to consider when designing or evaluating a through-year assessment program. The paper is organized into…

This article examines the concept of credible evidence in Extension evaluations with specific attention to the measures and measurement strategies used to collect and create data. Credibility depends on multiple factors, including data quality and methodological rigor, characteristics of the stakeholder audience, stakeholder beliefs about the…

The emergence of 21st-century skills convinced educational institutions in modifying teaching styles, which lead to the introduction of blended, flipped, and personalized learning. Studies demonstrated present-day best practices, but none provided a practical assessment procedure. Thus, this study aimed to develop a tool in evaluating learning…

Since 2003, the Oral History Project Team has conducted interviews with individuals who have made particularly noteworthy contributions to the theory and practice of evaluation. In 2013, Mel Mark, Valerie Caracelli, and Miles McNall sat with Thomas Cook in Washington, D.C., during the American Evaluation Association (AEA) annual conference. The…

The "NADE Self-Evaluation Guides" are a compendium of best practices in four areas: (1) Tutoring Services; (2) Course-based Learning Assistance; (3) the Teaching & Learning Process; and (4) Developmental Coursework, also recently known as transitional, co-requisite, accelerated, or bridge programs, all of which are…

Treatment effect estimates from a "regression discontinuity design" (RDD) have high internal validity. However, the arguments that support the design apply to a subpopulation that is narrower and usually different from the population of substantive interest in evaluation research. The disconnect between RDD population and the…

Teacher candidate performance assessments represent a promising source of data for evidence-based program improvement. However, teacher preparation programs (TPPs) interested in reform face a crucial question: how to identify actionable evidence in performance-assessment data. To address this concern, we propose a two-pronged empirical framework…

Programmatic assessment is both a philosophy and a method for assessment. It has been developed in medical education as a response to the limitation of the dominant testing or measurement approaches and to better align with changes in how medical competence was conceptualised. It is based on continual collection of assessment and feedback…

The purpose of this statement is to inform those using or considering the use of value-added models (VAM) about their scientific and technical limitations in the evaluation of educators and programs that prepare teachers. The statement briefly reviews the background and current context of using VAM for evaluations, enumerates specific psychometric…

This chapter proposes approaches for assessing learners and evaluating courses and curriculum that could be used by directors of health professions education (HPE) programs to determine the effectiveness and impact of their programs.