Identifying effective literacy interventions for teachers of very young children is important, but it is not easy. In this case study, we share our experiences implementing a quasi-experimental evaluation of an early literacy program. We describe the design and context of the study and discuss three key issues that we encountered with data…

Despite our best efforts as evaluators, program implementation failures abound. A wide variety of valuable methodologies have been adopted to explain and evaluate the "why" of these failures. Yet, typically these methodologies have been employed concurrently (e.g., project monitoring) or to the post-hoc assessment of program activities.…

This dissertation comprises three papers that propose, discuss, and illustrate models to make improved inferences about research questions regarding student achievement in education. Addressing the types of questions common in educational research today requires three different "extensions" to traditional educational assessment: (1)…

In the modern world, higher education is transitioning from enrollment mode to recruitment mode. This shift paved the way for institutional research and policy making from historical data perspective. More and more universities in the U.S. are implementing and using enterprise resource planning (ERP) systems, which collect vast amounts of data.…

The Office of Planning, Research and Evaluation (OPRE), a unit within the Administration for Children and Families (ACF), is responsible for advising the Assistant Secretary for Children and Families on increasing the effectiveness and efficiency of programs to improve the economic and social well-being of children and families. In collaboration…

This article reviews and analyzes the literature from 1980 through 1983 dealing with summative evaluation of training and development efforts in industry and business. An analysis is made to determine to what extent evaluation process components used by professional program evaluators are addressed by each of the articles reviewed. (CT)

In this article we report a 3-yr study of a large-enrollment Cell Biology course focused on developing student skill in scientific reasoning and data interpretation. Specifically, the study tested the hypothesis that converting the role of exams from summative grading devices to formative tools would increase student success in acquiring those…

This study explored some practical issues for single-case researchers who rely on visual analysis of graphed data, but who also may consider supplemental use of promising statistical analysis techniques. The study sought to answer three major questions: (a) What is a typical range of effect sizes from these analytic techniques for data from…

Many social workers are anxious about the use of evaluation techniques and are particularly reticent about using statistical research. Strategies for understanding the steps involved in integrating practice and evaluation in a school social work setting, an appreciation for the need of evaluation, and knowledge of various resources on conducting…

This third of three volumes reports on analytic procedures conducted to address various aspects of the scalar properties of the Device Effectiveness Forecasting Technique (DEFT). DEFT, a series of microcomputer programs applied to data gathered from rating scales, is used to evaluate simulator devices used in U.S. Army weapons training. The…

This paper examines issues in gathering data for purposes of making accountability decisions for students with disabilities. The first section describes five alternative methods being used: (1) administration of norm-referenced tests; (2) focus on results or outcomes; (3) Individualized Education Program aggregation; (4) secondary analysis of…

This report examines issues related to the performance indicators used for international comparisons of Australia's vocational education and training (VET) system. Chapter 2 examines main sources of international comparisons of VET and main features of indicator frameworks used for comparisons. Chapter 3 presents an outline of these two main…