This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

The evaluation plan for New York State's Children and Youth Intensive Case Management (CYICM) Program is described, which was introduced in July 1988. The CYICM Program is a statewide intervention focusing on keeping children with serious emotional disturbances in the least restrictive environment appropriate to their needs. It is a…

An evaluation system which would describe and assess statewide services for preschool children is described. Component theory conceptualizes the unit of analysis for evaluation as the component. This approach increases the generalizability and utilization of evaluations and enhances the ability to evaluate several programs at the state level.…

In response to 1986 state legislation calling for improvements in college effectiveness and accountability, the California Community Colleges began various efforts to promote the evaluation of matriculation. One effort, the Matriculation Local Research Options Project, was formed in 1989 to develop models for local evaluation of matriculation and…

In March 1989, the Northwest Regional Educational Laboratory entered into a contract with the Nevada Department of Education to develop this evaluation design for the Nevada Proficiency Examination (NPE) Program. The NPE program, mandated by state legislation, requires each school district to administer examinations in all public schools to…

The Guidelines for Educational Program Auditors in Utah were prepared to help educational program auditors (EPAs) deliver high quality services with a minimum of effort; they were intended to serve as a text for EPA training sessions, as a handbook and policy guide for EPAs and their assistants, and as a standard for judging the quality of EPA…

This report describes the technical assistance rendered to the NIE-sponsored R&D Utilization Program (R&DU) by the contractor working with the NIE staff, the Center for the Study of Evaluation, and the seven R&DU projects, to develop cross-project instrumentation, develop mutually acceptable plans for data collection and analysis, and…

This document presents the research proposal and supporting documents for a study to evaluate the process of developing a comprehensive dissemination capacity in State Education Agencies (SEAs) and to assess the impact of the capacity building activities in the states, addressing two questions: Is dissemination capacity being built and, if so,…

Design and implementation of a curriculum evaluation plan for the vocational education program (occupational education under the Carl D. Perkins Vocational and Applied Technology Act) of a large correctional system are described. An outside evaluator assisted in designing and conducting the first year's evaluation. The 1991-92 curriculum used was…

Plans are presented for the overall design of the longitudinal evaluation of the National Institute of Education's State Capacity Building Program. Both the process of developing comprehensive dissemination capacity in state education agencies, and the impact of the capacity building activities will be studied. This report describes the…

This primer is designed to provide a step-by-step guide for state policymakers and state agency officials interested in assessing the literacy skills of the people of their state. Chapter 1 describes the importance of assessing the literacy skills in each state. Literacy is discussed as an economic necessity, a requirement for a healthy democracy,…

This paper discusses a number of interventions that are aimed at increasing educational opportunities of homeless students in the United States. Only a few of these programs have been locally evaluated, and the evaluation process is particularly challenging for local evaluators to undertake. At a state and national level, commitment to evaluation…

This paper offers concrete examples from recent policy research about how qualitative and quantitative methods can be combined to better address research questions. Using a conceptual framework developed by Rossman and Wilson (1985), later expanded by Greene, Caracelli, and Graham (1989), the paper discusses how the design and analysis phases of…

A historical antagonism between the advocates of quantitative evaluation methods and the proponents of qualitative evaluation methods has stymied the recognition of the value to be gained by utilizing both methodologies in the same study. The integration of quantitative and qualitative methods within a single evaluation has synergistic effects in…

This study of evaluation utilization identified organizational, political, and practical arrangements facilitating the wide use of "The Interim Report Evaluation" in policymaking for California's Early Childhood Education Program. In a program fraught by tensions, where evaluations had been political tools, this evaluation was special.…