This response aims to critically evaluate the methodology and aims of the meta-analytic review written by Maggin and Johnson (2014). The present authors systematically provide responses for each of the original criticisms and highlight concerns regarding Maggin and Johnson's methodology, while objectively describing the current state of evidence…

Evaluators report effects of education initiatives as standardized effect sizes, a scale that has merits but obscures interpretation of the effects' practical importance. Consequently, educators and policymakers seek more readily interpretable translations of evaluation results. One popular metric is the number of years of learning necessary to…

Shortly after we published "Addressing Flawed Research in Developmental Education" (2012) in the "Journal of Developmental Education," Thomas Bailey, Shanna Smith Jaggars, and Judith Scott- Clayton from the Community College Research Center (CCRC) wrote a response rebutting several of our claims. Though their response corrects…

Implementation fidelity assessment provides a means of measuring the alignment between the planned program and the implemented program. Unfortunately, the implemented program can differ from the planned program, resulting in ambiguous inferences about the planned program's effectiveness (i.e., it is uncertain if poor results are due to an…

The growth in the availability of longitudinal data--data collected over time on the same individuals--as part of program evaluations has opened up exciting possibilities for evaluators to ask more nuanced questions about how individuals' outcomes change over time. However, in order to leverage longitudinal data to glean these important insights,…

Extension educators need simple and accurate evaluation tools for program evaluation. This article explains how to use aspiration as an outcome indicator in Extension evaluation and introduces a practical evaluation tool. Aspiration can be described as the readiness for change. By recording participants' levels of aspiration, we will be able to…

Background: Those involved in school health education programs generally believe that health-education programs can play an important role in helping young people make positive health decisions. Thus, it is to document the effects of such programs through rigorous evaluations published in peer-reviewed journals. Methods: This paper helps the…

Poisson and negative binomial regression methods are used to analyze the monthly time series data to determine the effects of introducing an integrated security management system including closed-circuit television (CCTV), door alarm monitoring, proximity card access, and emergency call boxes to a large privately-owned complex of apartment…

Establishing an online community of professional learners was one component of a multi-year professional development project with an overall goal of improving the ability of career and technical educators to use and interpret technical assessment data. Educators from five states and nine different schools participated in a research-based workshop…

This paper presents a framework for monitoring implementation which defines a process for implementing new programs into organizations. The process requires a team of monitors who examine the adequacy of implementation plans and look at the effect of plan execution upon the organization, individual, and new program. Immediate feedback is provided…

This paper explores a new approach to the evaluation of educational technologies by using Eisner's connoisseurship model. This model has two essential features in program evaluation: holistic approach to the analysis and interpretation of data and multiple perspectives in the evaluative tasks (Eisner, 1998). Using Eisner's model, the key…

While the "Standards for Evaluations of Educational Programs, Projects and Materials" provides a good checklist of prerequisites, such standards do not guarantee indisputable outcomes. Reanalysis of reading comprehension and mathematics mean achievement data from an international evaluation study illustrates how political bias can…

This document, which is intended for individuals at the state and local levels with responsibility for school-to-work opportunities, focuses on recommended standard procedures for evaluating school-to-work transition programs. In the introduction, the role of evaluation as a critical element in an overall program management strategy is discussed…

Evaluators usually assume that the data they receive are valid, or at least, that the data reflect that which is being evaluated. In a recent evaluation in a large southern school district, evaluators compared their raw interview notes with interview notes edited by the teachers interviewed or project staff. The program evaluated was Project SEED,…

In 1981, the developers of a school-based change effort engaged researchers to conduct an evaluation study employing qualitative methodology that would provide the developers with information that could be used to improve the program. Resource constraints, short timelines, and the developers' preference for useable information encouraged…