This paper presents a framework for utilizing Student Evaluations of Teaching (SET) in faculty evaluations. Recognizing the ongoing debate about the validity of SET as a measure of teaching effectiveness, the authors agree with scholars who propose viewing SET as a tool for gauging 'student perceptions of learning'. They present a method that…

The calls for accountability in higher education have made outcome-based assessment a key accreditation component. Accreditation remains a well-regarded seal of approval on college quality, and requires the programme to set clear, appropriate, and measurable goals and courses to attain them. Furthermore, programmes must demonstrate that…

In 1916 Robert Frost published his famous poem, "The Road Not Taken," in which he muses about what might have been had he chosen a different path, made a different choice. While counterfactual arguments in general can often lead to vacuous nowheres, frequently in statistics the data that are not presented actually exist, in a sense,…

Researchers use many different metrics for evaluation of performance of student models. The aim of this paper is to provide an overview of commonly used metrics, to discuss properties, advantages, and disadvantages of different metrics, to summarize current practice in educational data mining, and to provide guidance for evaluation of student…

The Office of Planning, Research and Evaluation (OPRE), a unit within the Administration for Children and Families (ACF), is responsible for advising the Assistant Secretary for Children and Families on increasing the effectiveness and efficiency of programs to improve the economic and social well-being of children and families. In collaboration…

In many practical settings, essentially the same differential item functioning (DIF) procedures have been in use since the late 1980s. Since then, examinee populations have become more heterogeneous, and tests have included more polytomously scored items. This paper summarizes and classifies new DIF methods and procedures that have appeared since…

Discusses a collaborative approach to evaluating professional training methods. Describes 4 objectives and 10 steps for "evaluating evaluation" that include identifying the training and evaluation goal, forming an evaluation team, designing the evaluation, developing the methods, collecting data, interpreting findings, and reporting results. (AEF)

Ethnographic study, which calls for tape-recorded interview data, requires a means of interpreting such data. One manner of interpretation is the fairly typical approach of providing descriptive and interpretive accounts of the persons who populate particular social scenes. Excerpts from or transcripts of recorded interviews are often used to…

A study involving 11,804 British children (age 10) found that when specified criteria for dyslexia were used, 269 children qualified as dyslexic. These included 223 boys and 46 girls, for a ratio of 4.51 to 1. Difficulties in interpreting these data are discussed and a defense of the criteria is provided. (Author/CR)

Analyzes two possible assessment outcomes: (1) differing results on two or more assessments; and (2) loss of achievement after improvement cycle. Proposes short- and long-term solutions to problematic results, including analyzing differences in tests and pre-planning to avoid flaws in assessment design. (NB)

Many social workers are anxious about the use of evaluation techniques and are particularly reticent about using statistical research. Strategies for understanding the steps involved in integrating practice and evaluation in a school social work setting, an appreciation for the need of evaluation, and knowledge of various resources on conducting…

Reports the rationale, design, implementation, and results of an examination of the impact of online subject catalogs on users by comparing their behaviors before and after catalog installation. Evaluation criteria included volume of use, types of searches, use of other catalogs, search time, user satisfaction, and need for assistance. (CLB)

A central dilemma of portfolio assessment is that as the richness of the data available to readers increases, so do the challenges involved in ensuring acceptable reliability among readers. Drawing on empirical and theoretical work in discourse analysis, ethnomethodology, and other fields, we argue that this dilemma results, in part, from the fact…

Data from nine Michigan elementary, middle, and high schools highlight the challenges and common strategies found in the use of student assessment data to meet public demands for accountability. The information gathered from assessment tools varies by frequency and type. Educators have, historically, underutilized data to guide practice because…

The Performance Review, Analysis, and Improvement System for Educators (PRAISE) is a formative evaluation instrument designed to improve the performance of school principals. The system appears to be reliable and valid and is flexible enough to accommodate the needs of a variety of schools. Sample items and categories of the instrument include…