In this article, we examine classroom observations from a 3-year large-scale randomized trial in the Los Angeles Unified School District (LAUSD) to investigate the extent to which a professional development initiative in inquiry science influenced teaching practices in in 4th and 5th grade classrooms in 73 schools. During the course of the study,…

This article reports on findings based on analyses of a unique dataset collected by ACT that includes information on student achievement in a variety of subjects at the high-school level. The authors examine the relationship between teacher effect estimates derived from value-added model (VAM) specifications employing different student learning…

Potential sources of bias are classified as ethical compromises, value inertias, or cognitive limitations. Thirteen specifics of intuitive thinking and judgmental processes are described. The author wishes to alert naturalistic evaluators to common failings. This list can be a useful checklist in reducing, integrating and drawing inferences from…

Drawing on British data, this article discusses issues involved in analyzing educational indicator data. Gross productivity, new productivity, and inequality are discussed as types of indicators that describe interdistrict variation in Great Britain. Recommendations are made regarding features of an educational assessment that would be necessary…

The Judicial Evaluation Method (JEM) is useful for aiding decision-making bodies to formulate policies. Broad public participation in complex issues is provided by JEM. (The conceptualization, policy formulation stages, and participant roles are identified; and the model is applied in two case studies). (MH)

The current literature is replete with instances of imprecise and conflicting usage of "qualitative" and "quantitative" evaluation. This paper presents a framework for making distinctions among several related concepts to eliminate this potential source of confusion. In the process, the terms "qualitative" and…

The following four integrative data analysis strategies for mixed-method evaluation designs are derived from and illustrated by empirical practice: (1) data transformation; (2) typology development; (3) extreme case analysis; and (4) data consolidation and merging. Use of these methods to realize the full potential of mixed-method approaches is…

Examines and compares various outcomes of education, such as standardized test results, classroom observations, teacher reports, and teacher interviews, in terms of their potential to inform policymakers about policy influences on student outcomes. Suggests the need for clearer understanding of how these approximations are related to each other.…

A critical guide is offered regarding ethnographic evaluation data: the analytical descriptions or reconstructions of intact cultural scenes and groups which delineate the shared beliefs, practices, artifacts, folk knowledge, and behaviors of some group of people. (PN)

A mixed-method conceptual framework was developed from the literature and refined through an analysis of 57 empirical mixed-method evaluations. Five purposes for mixed-method evaluation are identified (triangulation, complementarity, development, initiation, and expansion) and analyzed using the framework of 7 design characteristics. (TJH)

This paper describes eight problems faced by a school district evaluator in evaluating state-funded, compensatory education programs: determining evaluation ownership, determining the major constraints, determining what to evaluate, determining what methods to use, determining participant characteristics, determining instrumentation, determining…

A set of methods is proposed for the analysis of opportunity to learn (OTL) in relation to achievement in large-scale educational assessments. Methods are illustrated with data from the National Assessment of Educational Progress and the National Education Longitudinal Study. Implications for large-scale educational assessment are discussed. (SLD)

Problems that face evaluators, including field based evaluations, are discussed. The author states that open communication, access to data, and an ability to be accepted are paramount; knowledge of randomness, robustness and homoscedasticity may be secondary. (Author/PN)

Validation studies, placement-enrollment comparisons, student surveys, and instructor interviews are discussed as operational methods in a synthesizing approach to undergraduate course placement evaluation. A range of suggested questions is coupled with various data sources in placement evaluation schemes and system procedures. (CM)

A communication theory paradigm typically is used to investigate the relationship between evaluation information and utilization: "Who says what, how, to whom, with what effects?" This study investigates the "how," or medium of presentation, aspect of the paradigm. (Author/LMO)