Power has always been an element of the program evaluation field, however, it has traditionally been attributed a negative connotation. With that in mind, power is seen as a force possessed to some extent by different stakeholders that can be imposed over others, and by which certain views, ideas, and decisions can be adopted. Given that program…

Over the last decade, the idea of societal impact resulting from publicly funded research has changed from being a relatively fringe concern related to high-technology entrepreneurship and spin-off companies to becoming an increasingly important public policy concern. This has stimulated academic science policy research to investigate the impact…

Great opportunities for language testing practitioners are enabled through language program administration. Local language tests lend themselves to multiple purposes--for placement and diagnosis, as a means of tracking progress, and as a contribution to program evaluation and revision. Administrative choices, especially those involving a test, are…

This method note presents Q methodology as a useful tool for evaluators to add to their practice toolbox. Q methodology, which involves both quantitative and qualitative techniques, can help researchers and evaluators systematically understand subjectivity and the communicability of opinions and perspectives. We first provide an overview of Q…

In this practice paper I examine the evaluation of one project and ask whether or not the evaluation failed. I suggest that because I was unaware of the larger connections between the grant agency and the grantee I made tactical errors in the presentation of findings, which resulted in not simply disagreement but anger and resentment. I conclude…

This article presents a new tool called Critical Evaluation Capital (CEC) designed to address issues of equity and social justice in program evaluation. CEC is grounded in the tenants of critical race theory and inspired by Yosso's work on community cultural wealth which raises critical issues of positionality and access. CEC is a system for…

Recent public events have unnerved the country, touched the consciousness of the populace, and reveal the resurgence of the White supremacist movement, causing many citizens to reflect on race relations. Given the profound impact of racism deeply embedded in the fabric of this country and its complex and destructive influence on the lives of…

This article explores issues that had to be confronted over a three-year period in terms of two New York State funded grants for which I was the evaluator. Major, critical challenges are presented, described, and discussed. Specifically, the paper addresses issues related to lack of planning at the grant design stage that resulted in challenges…

Many studies of education interventions make claims about impacts on students' outcomes. Some studies have designs that enable readers to make causal inferences about the effects of an intervention but others have designs that do not permit these types of conclusions. To help policymakers, practitioners, and others make sense of study results, the…

The importance of engaging in high quality program evaluation is a generally accepted principle underscored by external pressure from funders. High quality evaluation necessarily begins with good evaluation planning. This paper outlines Evolutionary Evaluation and specifically the Systems Evaluation Protocol, an approach that emphasizes…

Funders and policymakers are increasingly recognizing the afterschool field for its vital role in supporting the social and emotional growth and academic achievement of school-age youth. Although this recognition is welcome, it often comes with increased expectations for high-quality research demonstrating the value of programming. To satisfy…

The What Works Clearinghouse (WWC) evaluates research studies that look at the effectiveness of education programs, products, practices, and policies, which the WWC calls "interventions." Many studies of education interventions make claims about impacts on students' outcomes. Some studies have designs that enable readers to make causal…

In this article the authors present the full interview conducted with George Grob in 2011 at the American Evaluation Association (AEA) Conference in Anaheim, CA. George Grob is former Director of the Office of Evaluation and Inspections in the Office of Inspector General. Prior to serving in that Office, he was Director of Planning and Policy…

Successive UK government policies have strengthened the phonics element of the National Curriculum for English in England. The policies have included inviting publishers to submit completed self-assessments of their systematic phonics programmes. The self-assessment criteria focus on what is deemed to be 'high-quality provision', as defined in the…

This article provides a framework to assist evaluators, teaching artists, and program managers in working together.