Program evaluation is an essential practice for providers of behavior analytic services, as it helps providers understand the extent to which they are achieving their intended mission to the community they serve. A proposed method for conducting such evaluations, is through the use of a consecutive case series design, for which cases are…

As is the case with many Centres for Teaching and Learning, prior to the global health crisis in 2020 our Centre primarily obtained feedback to inform its programming based on participation and satisfaction. As a result of the crisis, we could not rely on previous sources of information and needed to implement new strategies for evidencing the…

Purpose: The firms use training evaluation practices (TEPs) to determine the return of billions of dollars spent on employee training and development activities. The firms need to modernize the set of TEPs for evidence-based workforce management decisions. This study aims to examine a mediation mechanism to explain how human resource (HR)…

Analysis of symmetrically predicted endogenous subgroups (ASPES) is an approach to assessing heterogeneity in an ITT effect from a randomized experiment when an intermediate variable (one that is measured after random assignment and before outcomes) is hypothesized to be related to the ITT effect, but is only measured in one group. For example,…

The purpose of this paper is to introduce the Five-Level Qualitative Data Analysis (5LQDA) method for ATLAS.ti as a way to intentionally design methodological approaches applicable to the field of evaluation. To demonstrate my analytical process using ATLAS.ti, I use examples from an existing evaluation of a STEM Peer Learning Assistant program.…

Program evaluation is important for assessing the implementation and outcomes of local, state, and federal programs. The Program Evaluation Toolkit provides tools and resources to support individuals responsible for evaluating and monitoring local, state, or federal programs. The toolkit comprises eight modules that cover critical steps in program…

Policymakers have expressed increased interest in program-level higher education accountability measures as a supplement to, or in place of, institution-level metrics. But it is unclear what these measures should look like. In this report, we assess the ways program-level data could be developed to facilitate federal accountability. Evidence shows…

This article examines the concept of credible evidence in Extension evaluations with specific attention to the measures and measurement strategies used to collect and create data. Credibility depends on multiple factors, including data quality and methodological rigor, characteristics of the stakeholder audience, stakeholder beliefs about the…

Several analytic strategies exist for opening up the "black box" to reveal more about what drives policy and program impacts. This article focuses on one of these strategies: the Analysis of Symmetrically-Predicted Endogenous Subgroups (ASPES). ASPES uses exogenous baseline data to identify endogenously-defined subgroups, keeping the…

Extension program evaluations often present opportunities to analyze data in multiple ways. This article suggests that program evaluations can involve more sophisticated data analysis approaches than are often used. On the basis of a hypothetical program scenario and corresponding data set, two approaches to testing for evidence of program impact…

Interest in the regression discontinuity (RD) design as an alternative to randomized control trials (RCTs) has grown in recent years. There is little practical guidance, however, on conditions that would lead to a successful RD evaluation or the utility of studies with underpowered RD designs. This article describes the use of RD design to…

This article presents a method for using mind mapping to assess social learning outcomes in collaborative environmental restoration and participatory natural resource management initiatives. Using mind mapping for preassessment and postassessment can reveal changes in individual and collective thinking about critical social and ecological issues.…

A mind mapping approach to evaluation called Ripple Effects Mapping (REM) has been developed and used by a number of Extension faculty across the country recently. This article describes three approaches to REM, as well as key differences and similarities. The authors, each from different land-grant institutions, believe REM is an effective way to…

Prior Learning Assessment (PLA) is the evaluation and assessment of learning gained outside a traditional academic environment for college credit, certification, or advanced standing toward further education or training. The term PLA encompasses a wide spectrum of assessment methods, including, but not limited to, credit by examination, challenge…

This study described the education courses in Master's Degree and Post-master's Certificate in nursing education programs and determined the extent to which the eight core competencies, used to certify nurse educator's, were included. The data regarding the required credit hours, practicum hours, distance accessibility, and preparation for the…