Institutional researchers are often tasked with assessing why college-wide initiatives succeed or fail. This can be a difficult task: researchers need to discriminate between multiple feasible explanations, work with limited data, and produce compelling narratives. Process tracing is a qualitative methodology that enables researchers to make valid…

This chapter proposes approaches for assessing learners and evaluating courses and curriculum that could be used by directors of health professions education (HPE) programs to determine the effectiveness and impact of their programs.

Several recent studies have shown that children with autism spectrum disorder (ASD) are less physically active than their typically developing peers and are thus at risk of health problems associated with inactivity (e.g., obesity or diabetes). There is a need to examine the effectiveness of interventions such as swimming that are aimed at…

Evaluations should be issues driven, not methods driven. The starting point should be priority programs to be evaluated or policies to be tested. From this starting point, a list of evaluation questions is identified. For each evaluation question, the task is to identify the best available method for answering that question. Hence it is likely…

Audience response systems (ARS) are gaining popularity in multiple areas of education. Although its usefulness has been well reported, no study has investigated the accuracy of the technology. The purpose of this study was to validate ARS by assessing three key aspects: signal transmission from the keypad to the receiver, capture of a proper…

This chapter presents a number of problems related to designing local-level summative evaluation studies and suggests solutions to these problems. Three general areas are covered. First, many of the motivational, organizational, and attitudinal constraints likely to face the local-level evaluator are discussed. Second, various threats to the…

Factors involved in pre-planned and post-planned evaluation of program effectiveness are compared: (1) reliability and cost of data; (2) internal and external validity; (3) obtrusiveness and threat; (4) goal displacement and program direction. A model to help program administrators decide which approach is more appropriate is presented. (Author/MH)

This article presents the results of a study conducted in Anchorage, Alaska, that evaluated the accuracy and efficiency of using voice recognition (VR) technology to collect oral reading fluency data for classroom-based assessments. The primary research question was as follows: Is voice recognition technology a valid and reliable alternative to…

City-as-School (CAS), founded in 1972, is a New York City alternative high school with a mission to link students to learning experiences outside the classroom. Since the 1985-86 school year, CAS has received federal funds for the CAS Replication Project, the goal of which is to promote and facilitate the adoption of the CAS model by other sites.…

This report summarizes the National Job Training Partnership Act (JTPA) Study, which examined the impact of Title IIA of JTPA on participants' employment, earnings, and welfare receipt. Between 1987 and 1989, more than 20,000 adults and out-of-school youths who applied for JTPA services in 16 areas across the country were randomly assigned to a…

A study examined the information collected by the Department of Defense on both the quality of its new recruits and the effectiveness of its training in preparing recruits to operate in a technologically sophisticated environment. It found that data were collected at a recruit's entrance to military life, during and upon completion of formal…

The orientation of this report is that of identifying educational projects which can be considered clearly exemplary. The largest section consists of a 22-step procedure for validating the effectiveness of educational projects using existing evaluation data. It is not intended as a guide for conducting evaluations but rather for interpreting data…