The investigation focused on the effects of using grouped data to estimate the relations that exist in data on individuals. Different research contexts were identified in which researchers group observations though interested in relations among measurements on individuals. The consequences of estimating regression coefficients from grouped data…

The purpose of this paper is to state the reasons why secondary analysis serves an important function in educational research. (Author)

Problems associated with the analysis and interpretation of aggregated data in school effects research are discussed. The multilevel character of school effects research is described. Data from the IEA Six-Subjects Survey of Science Achievement in Australia are used to illustrate the differences in between-student and between-school models of the…

Two specific methods of analysis in large-scale evaluations are considered: structural equation modeling and selection modeling/analysis of non-equivalent control group designs. Their utility in large-scale educational program evaluation is discussed. The examination of these methodological developments indicates how people (evaluators,…

The activities of the Multilevel Analysis Project of the Center for the Study of Evaluation from February 1 to June 1, 1978 are summarized. Twelve project staff activities are listed, including papers on alternative analytical models for identifying educational effects, conference presentations, and grant proposals. A report on empirical…

Educational outcomes are affected by student level, classroom level, and school level characteristics. The fact that educational data are multilevel in nature poses serious analysis questions. Though strong arguments can be made for focusing on a single level of analysis, such studies have several basic limitations: the choice of analytic level…

This discussion of how the multilevel character of data should influence analyses of the determinants of educational achievement is grounded in the context of planning explanatory analyses for the longitudinal Second International Mathematics Study (SIMS). The intention is to describe how a multilevel perspective in the specification of…

Since problems associated with the statistical methodology of educational research are becoming increasingly important, this paper examines a subset of problems associated with the analysis and interpretation of aggregated data. Two major questions arise: (1) if a researcher knows the level (e.g., individual, teacher/classroom, school, school…

The purpose of this report is to comment on practice in the use of existing data bases in program evaluation and school improvement and to explore directions of increased and improved use. The report's impetus is the role that existing data plays in current efforts in local school districts and the possibilities for the future. To a great extent,…

This is one of a series of reports based on an ongoing reality test of systemic evaluation for instructional decision making. This feasibility study is being carried out by the Center for the Study of Evaluation with the Laboratory in School and Community Relations at a suburban Los Angeles high school (called Site A). Viewing a school as a…

A conceptual synthesis is presented of the ways in which multilevel methods of data collection, analysis, interpretation and reporting can be used in instructional improvement activities in local school districts. This discussion of multilevel methods in school settings focuses on the "what" of information needs and usage. Principles and…

Instructional and program relevance of measures of student performance and use of standardized achievement tests, as measures of the quality of a student's educational experiences, are discussed. This focus is consonant with current emphasis on linking testing and instruction and on systemic efforts at program and instructional improvement.…

Multiple sources of complications in the analysis of multilevel educational data are described with emphasis on the determination of a method of analysis that will provide accurate estimates of teacher/class effects even when there are systematic differences in within-class regressions of outcome on input associated with teacher/class…

The author contends that the secondary analyst may have a different perspective and theoretical persuasion than the original investigator, may have the advantage of more current theory and practices than those originally available, may move beyond the substantive and methodological limitations of the original investigation, and may not be faced by…

Activities of the California State Department of Education related to the Early Childhood Education Program were audited, specifically: (1) overall data management and processing; (2) information quality for the evaluation of program impact; (3) treatment of standardized test data; and (4) procedures for expansion funding and termination. (CP)