Program evaluation is an essential practice for providers of behavior analytic services, as it helps providers understand the extent to which they are achieving their intended mission to the community they serve. A proposed method for conducting such evaluations, is through the use of a consecutive case series design, for which cases are…

As is the case with many Centres for Teaching and Learning, prior to the global health crisis in 2020 our Centre primarily obtained feedback to inform its programming based on participation and satisfaction. As a result of the crisis, we could not rely on previous sources of information and needed to implement new strategies for evidencing the…

Purpose: The firms use training evaluation practices (TEPs) to determine the return of billions of dollars spent on employee training and development activities. The firms need to modernize the set of TEPs for evidence-based workforce management decisions. This study aims to examine a mediation mechanism to explain how human resource (HR)…

Analysis of symmetrically predicted endogenous subgroups (ASPES) is an approach to assessing heterogeneity in an ITT effect from a randomized experiment when an intermediate variable (one that is measured after random assignment and before outcomes) is hypothesized to be related to the ITT effect, but is only measured in one group. For example,…

The purpose of this paper is to introduce the Five-Level Qualitative Data Analysis (5LQDA) method for ATLAS.ti as a way to intentionally design methodological approaches applicable to the field of evaluation. To demonstrate my analytical process using ATLAS.ti, I use examples from an existing evaluation of a STEM Peer Learning Assistant program.…

BRIDGES is a software framework for creating engaging assignments for required courses such as data structures and algorithms. It provides students with a simplified API that populates their own data structure implementations with live and real-world data, and provides the ability for students to easily visualize the data structures they create as…

Background: Mixed methods research designs in engineering education often frame the "mixed" aspect of the design from the perspective of data collection. However, intentional mixing throughout the research design--particularly during data analysis--may enable richer meta-inferences about a phenomenon. Purpose: This paper provides…

Program evaluation is important for assessing the implementation and outcomes of local, state, and federal programs. The Program Evaluation Toolkit provides tools and resources to support individuals responsible for evaluating and monitoring local, state, or federal programs. The toolkit comprises eight modules that cover critical steps in program…

Policymakers have expressed increased interest in program-level higher education accountability measures as a supplement to, or in place of, institution-level metrics. But it is unclear what these measures should look like. In this report, we assess the ways program-level data could be developed to facilitate federal accountability. Evidence shows…

Even under ideal conditions, teaching is tough work. Facing unrelenting pressure from administrators and parents and caught in a race against time to improve student outcomes, educators can easily become discouraged (or worse, burn out completely) without a robust coaching system in place to support them. For more than 20 years, perfecting such a…

This article examines the concept of credible evidence in Extension evaluations with specific attention to the measures and measurement strategies used to collect and create data. Credibility depends on multiple factors, including data quality and methodological rigor, characteristics of the stakeholder audience, stakeholder beliefs about the…

Attention to mixed methods studies research has increased in recent years, particularly among funding agencies that increasingly require a mixed methods approach for program evaluation. At the same time, researchers operating within large-scale, rapid-turnaround research projects are faced with the reality that collection and analysis of large…

The "Forum Guide to Data Literacy" is designed to help education agencies understand and build data literacy skills among various stakeholder groups such as administrators, teachers, students, parents and other caregivers, school board members, legislators, and community groups. This resource defines and discusses the importance of data…

Participation of people with disabilities in producing and using research is critical for monitoring the implementation of the United Nations Convention on the Rights of Persons with Disabilities (CRPD). The Research Active Programme (RAP) is a module designed to build research capacity of students with intellectual disabilities. RAP was…

This article presents discussion and recommendations on approaches to retrospectively evaluating development interventions in the long term through a systems lens. It is based on experiences from the implementation of an 18-month study to investigate the impact of development interventions on economic and social change over a 40-year period in the…