Background: Traditional survey efforts to gather outcome data at scale have significant limitations, including cost, time, and respondent burden. This pilot study explored new and innovative large-scale methods of collecting and validating data from publicly available sources. Taking advantage of emerging data science techniques, we leverage…

This article examines the concept of credible evidence in Extension evaluations with specific attention to the measures and measurement strategies used to collect and create data. Credibility depends on multiple factors, including data quality and methodological rigor, characteristics of the stakeholder audience, stakeholder beliefs about the…

Objectives: In the USA, 18% of school-aged young people are classified as obese, and rural populations appear to be particularly at risk. Achieving high levels of fitness reduces the risk of obesity and underlying health conditions. To better understand youth obesity trends and fitness levels, annual fitness testing ([FT], that is, surveillance)…

This chapter proposes approaches for assessing learners and evaluating courses and curriculum that could be used by directors of health professions education (HPE) programs to determine the effectiveness and impact of their programs.

Purpose: This paper aims to briefly outline the seamless evaluation approach and its application during an evaluation of ORIENT, a serious game aimed at young adults. Design/methodology/approach: In this paper, the authors detail a unobtrusive, embedded evaluation approach that occurs within the game context, adding value and entertainment to the…

Over the past few years, the focus group method has assumed a very important role as a method for collecting qualitative data in social and behavioural science research. This article elucidates theoretical and practical problems and prospects associated with the use of focus groups as a qualitative research method in social and behavioural science…

Teacher preparation programs (TPPs) are where prospective teachers gain a foundation of knowledge about pedagogy and subject matter, as well as early exposure to practical classroom experience. Although competence in teaching, as in all professions, is shaped significantly by on-the-job experiences and continuous learning, the programs that…

This handbook is intended to help educational evaluators use still photography in designing, conducting, and reporting evaluations of educational programs. It describes techniques for using a visual documentary approach to program evaluation that features data collected with a camera. The emphasis is on the aspects of educational evaluation…

Thirty studies on supported employment for people with disabilities were evaluated using a proposed Social Validity Matrix. Results suggested further research should investigate alternative strategies for assessing social validity, develop systematic procedures for collecting and using social validity data, and establish functional variables…

This paper proposes a model to establish which criteria are considered by stakeholders as valid for evaluating a program. The model is developed with the aim of increasing the credibility and use of evaluations. Stakeholders are involved in the identification of potential evaluation criteria, and ratings of validity and priority are used as the…

The linked judgments approach is offered as a potential solution to problems of statistical conclusion validity when judges' ratings are employed in pretest-posttest designs. (Author/GK)

Discusses the quality of school-level data collected as part of state accountability systems, including the reliability and validity of school-level test scores as a measure of the value added by schools to student learning, outlining ways that these data can be usefully analyzed; illustrating challenges inherent in doing so; and discussing…

A Lesson Plan Evaluation Form was developed to provide systematic quantitative data about classroom functioning in an elementary-level gifted program. Plans (n=137) from 15 teachers were analyzed, and useful information about program functioning was obtained in a cost-effective manner. Data obtained were consistent with data from more traditional…

This chapter presents a number of problems related to designing local-level summative evaluation studies and suggests solutions to these problems. Three general areas are covered. First, many of the motivational, organizational, and attitudinal constraints likely to face the local-level evaluator are discussed. Second, various threats to the…

Factors involved in pre-planned and post-planned evaluation of program effectiveness are compared: (1) reliability and cost of data; (2) internal and external validity; (3) obtrusiveness and threat; (4) goal displacement and program direction. A model to help program administrators decide which approach is more appropriate is presented. (Author/MH)