Analysis of symmetrically predicted endogenous subgroups (ASPES) is an approach to assessing heterogeneity in an ITT effect from a randomized experiment when an intermediate variable (one that is measured after random assignment and before outcomes) is hypothesized to be related to the ITT effect, but is only measured in one group. For example,…

This article examines the concept of credible evidence in Extension evaluations with specific attention to the measures and measurement strategies used to collect and create data. Credibility depends on multiple factors, including data quality and methodological rigor, characteristics of the stakeholder audience, stakeholder beliefs about the…

This article provides insight into how an adequate approach to selecting methods can establish credible and actionable evidence. The authors offer strategies to effectively support Extension professionals, including program developers and evaluators, in being more deliberate when selecting appropriate qualitative and quantitative methods. In…

Several analytic strategies exist for opening up the "black box" to reveal more about what drives policy and program impacts. This article focuses on one of these strategies: the Analysis of Symmetrically-Predicted Endogenous Subgroups (ASPES). ASPES uses exogenous baseline data to identify endogenously-defined subgroups, keeping the…

This article is a call for research to be conducted on how to adequately design and evaluate outdoor leader preparation programs (Sugerman, 1999). The profession of outdoor leadership has been slow to examine effective ways of assessing the development of student knowledge, disposition, and performance that inform instructional practice and…

The authors explored the credibility of using informal reading inventories and writing samples for 138 students (K-4) to evaluate the effectiveness of a summer literacy program. Running Records (a measure of a child's reading level) and teacher experience during daily reading instruction were used to estimate the reliability of the more formal…

The relevance of ethnographic research to evaluation of educational programs is discussed. The author focuses on evaluation questions that deal with finding out what is happening in a program and if what is happening is desirable. Ethnography suits the purposes of evaluators who answer such questions because they want to know how something works…

Chelimsky, former head of the Program Evaluation and Methodology Division of the General Accounting Office, suggested that advocacy by evaluators destroys their credibility. Evaluators should, this author argues, be advocates for democracy and the public interest, with the question being how explicitly and how defensibly. (SLD)

Still photography can help portray educational programs for description and evaluation. There may, however, be problems with this method. Photographic images idealize the subjects; the framing is arbitrary, lends importance and veracity at the same time it levels an event to a very thin slice of reality. The evaluator-photographer may have…

Selected aspects of the problem of obtaining unbiased program or product evaluation are discussed. An evaluator who is a member of the project staff will have difficulty producing an evaluation which is credible and valid. Project monitors will also have a problem since they are often required to assume the conflicting roles of external evaluator…

Investigated two variance estimation procedures on a normed test treated as a criterion-referenced test. The first estimation procedure estimated the standard deviation systematically higher, whereas the second procedure's estimation was systematically lower than the actual standard deviation. (Author/RC)

Provides guidelines on how state and local administrators can develop an accountability system to meet Carl D. Perkins Vocational and Applied Technology Education Act requirements. Indicates that performance measures and standards should (1) be clearly defined; (2) be manageable; (3) include accurate and timely data; and (4) provide information…

Four key entities participating in foreign assistance projects are (1) donors; (2) technical advisors; (3) host country counterparts; and (4) beneficiaries. Resistance to collaborative evaluation that each may exhibit is discussed, along with ways evaluators can overcome resistance and enhance evaluation outcomes, credibility, and utilization.…

Evaluation is viewed as essential to decision making and social policy development. Since conventional methods have been disappointing or inadequate, naturalistic inquiry (N/I) differs from conventional science in minimizing constraints on antecedent conditions (controls) and on output (dependent variables). N/I is phenomenological rather than…

The political context of evaluation is described, with positive and negative influences. Strategies are proposed for preserving the credibility of evaluation in political contexts. Researchers must maintain the neutral posture of the basic scientist. (SLD)