The Campbellian validity typology has been used as a foundation for outcome evaluation and for developing evidence-based interventions for decades. As such, randomized control trials were preferred for outcome evaluation. However, some evaluators disagree with the validity typology's argument that randomized controlled trials as the best design…

Premised on the idea that evaluators should be familiar with a range of approaches to program modifications, I review several existing approaches and then describe another, less well-recognized option. In this newer option, evaluators work with others to identify potentially needed adaptations for select program aspects "in advance." In…

Policymakers have expressed increased interest in program-level higher education accountability measures as a supplement to, or in place of, institution-level metrics. But it is unclear what these measures should look like. In this report, we assess the ways program-level data could be developed to facilitate federal accountability. Evidence shows…

As part of its regulatory roles, the Australian Government Tertiary Education Quality and Standards Agency (TEQSA) is required to assess evidence of 'scholarship' that is put forward by providers, whether evidence of scholarship by individuals, such as teaching staff within the provider, or for the provider as a whole. TEQSA is seeking to review…

Duplicating processes and procedure with anticipation of deviating outcomes is the defining trait of insanity as attributed to a quote by Albert Einstein. It is the antithesis to innovation, which is what is needed in higher education to create impactful changes in the outreach we should be providing to the community. What is important for those…

The author's aim is to explore the school principal training and development landscape in Macau. The continuities and changes in the professional development of principals in Macau over the last two decades are identified. Using Huber's theoretical framework, key pre- and in-service policy documents from 1993 to 2017 that have shaped the…

Most states have developed or revised school accountability systems in response to the Every Student Succeeds Act (ESSA). While these systems include multiple indicators, to many stakeholders the outcome of central interest is the overall rating or classification produced for each school. These ratings are often used to identify schools that merit…

Classification evaluation metrics are often used to evaluate adaptive tutoring systems-- programs that teach and adapt to humans. Unfortunately, it is not clear how intuitive these metrics are for practitioners with little machine learning background. Moreover, our experiments suggest that existing convention for evaluating tutoring systems may…

One prevention strategy for childhood sexual abuse (CSA) involves educational programs delivered to children in the school environment. The purpose of this integrative literature review was to determine the state of the science on school-based CSA prevention programs. The authors extracted data from 26 articles that fit inclusion criteria to…

This research investigates the reliability and validity of three major publications' rankings of MBA programs. Each set of rankings showed reasonable consistency over time, both at the level of the overall rankings and for most of the facets from which the rankings are derived. Each set of rankings also showed some levels of convergent and…

The act of valuing in an evaluation may be perceived in different ways. We consider the multiple theoretic perspectives that govern an evaluator's behavior and present a typology of evaluator valuing roles. Within this typology we describe three ways in which value judgments are typically reached--by stakeholders alone, stakeholders and evaluators…

The debate surrounding teacher quality often fails to differentiate effectively between teacher-preparation providers. This failure also extends to distinguishing between teachers prepared in traditional campus-based accredited programs from those prepared in unaccredited campus-based programs. This paper compares assessment infrastructure and…

This brief (1) defines indicators, (2) places the concept into the context of the various ways indicators can be used in education, (3) explores some specific considerations and concerns that arise in evaluating results, (4) offers a categorization and examples of short-term outcome indicators for school use, and (5) stresses the need for policy…

This "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)" provides a detailed description of the standards and procedures of the What Works Clearinghouse (WWC). The remaining chapters of this Handbook are organized to take the reader through the basic steps that the WWC uses to develop a review protocol, identify…

This article reviews recent evaluation studies of online learning communities to provide a systematic understanding of how such communities are evaluated. Forty-two representative studies were selected and categorized into a newly developed taxonomy of online learning community evaluations. This taxonomy is divided into four components: evaluation…