Objective: Our master's program in biostatistics requires a qualifying examination (QE). A curriculum review led us to question whether to replace a closed-book format with an open-book one. Our goal was to improve the QE. Methods: This is a case study and commentary, where we describe the evolution of the QE, both in its goals and its content.…

This small-scale, mixed-methods study aims to investigate academics' understanding of formative and summative assessment methods and how assessment literacy impacts on their teaching methods. Six semi-structured interviews and a scrutiny of assessments provided the data and results suggest that while these academics understand summative…

Maxwell, Cole, and Mitchell (2011) demonstrated that simple structural equation models, when used with cross-sectional data, generally produce biased estimates of meditated effects. I extend those results by showing how simple structural equation models can produce biased estimates of meditated effects when used even with longitudinal data. Even…

This article challenges Yaakov Kareev's (1995a, 2000) argument regarding the positive bias of intuitive correlation estimates due to working memory capacity limitations and its adaptive value. The authors show that, under narrow window theory's primacy effect assumption, there is a considerable between-individual variability of the effects of…

Documents the process by which grades are assigned in a centrally-controlled, multiple-section course. Results of participant observation indicate that grades are determined by organizational processes instead of abilities or performance of either students or instructors. Concludes that knowledge of the rating process is essential to understanding…

Formative feedback systems live at the heart of systemic school improvement efforts. Without accurate and timely information on the results of intended interventions, school leaders and teachers fly blind in their efforts to link what they expect to what actually happens in their schools. This paper draws on the results of a 5-year National…

Policy analysis consists of evaluating policy options in terms of their effects. The analysis is not complete, however, until the reasoning has been reversed, starting at the outcome end and reasoning back to the first choice (backward mapping). Reversing the logic has two effects on analysis; it provides insurance against unanticipated effects,…

The derivation of a general retrieval effectiveness measure is investigated. Inconsistencies in the definition of relative importance are resolved, and it is shown that the influence of the assigned weight on the effectiveness measure can produce counterintuitive results. An alternative effectiveness measure provided by the Marczewski-Steinhaus…

The assumption of equal treatment implementation is questioned. Through the reanalysis of data from a nutrition supplementation program evaluation, the power of the analysis of treatment effects is shown to increase when data on the level of treatment implementation is included. (Author/CM)

Examines a series of measures often used to study inequality of personal income and applies the more theoretically acceptable measures to per pupil expenditure data for Ontario school boards during the period 1965 to 1976. (Author/IRT)

The curriculum implementation models reviewed utilize observation techniques, focused interviews, questionnaires, and content analysis. Concludes that determining the relationship between the level of program implementation and program outcomes prevents curriculum from becoming ineffective. (KC)

Discusses the concept of managers as coaches and summarizes a study that was conducted to evaluate coaching skills among sales managers by using telephone interviews with employees to collect data regarding managers' effectiveness in eight coaching skills. Behaviors associated with high and low rankings are identified. (Contains 26 references.)…

Survey of career and employment counseling showed that little counseling evaluation was being done in schools, colleges, universities, community agencies, or Canada Employment Centres. Absence of evaluative information places service at risk of being discontinued. Those in leadership positions have to be convinced of need for more and better…

Describes the evaluation of a new model for software evaluation that emphasizes the collection of student performance data to determine the extent to which students learn the knowledge a software package intends to teach. Discussion covers the results of two studies, evaluation criteria and methods, evaluator subjectivity in decision making,…

A problem, endemic to program evaluation, centering around the inability of evaluations to produce meaningful information, is outlined. A general strategy to be utilized in responding to the problem, the ethnographic approach, is presented. The discussion of an ethnographic evaluation model focuses upon: the role of the participant observer,…