This study investigates the effect of using rubrics as an instructional tool on students' writing performance in English as a foreign language. The major goal was to liberate the students from the narrow perception of writing based solely on the notion of correct grammar which is strongly felt in EFL writing. In this quasi-experimental research,…

In recent years, school-based assessment (SBA) has been incorporated into the English Language subject of a traditional high-stakes public examination, the Hong Kong Certificate of Education Examination. As reactions from various stakeholder groups have been mixed, it was necessary to review this new practice. This paper reports on a study of 33…

Although the importance of addressing and evaluating treatment and program fidelity is clearly emphasized in the literature on psychology, education, and health, little attention has been given to fidelity in adventure literature or research. Program fidelity refers to whether or not, and how well, a specific intervention or program was…

Background: Evaluation is sometimes viewed as a professional practice rather than a discipline corresponding to a well defined set of theories. However, Shadish, Cook and Leviton (1991) were able to demonstrate that evaluators' work does have theoretical foundations. In particular, the authors identified five main elements for evaluation theory…

Regressions that control for confounding factors are the workhorse of evaluation research. When treatment effects are heterogeneous, however, the workhorse regression leads to estimated treatment effects that lack behavioral interpretations even when the selection on observables assumption holds. Regressions that use propensity scores as weights…

This report presents work on the development of a new corpus of non-native English writing. It will be useful for the task of native language identification, as well as grammatical error detection and correction, and automatic essay scoring. In this report, the corpus is described in detail.

Background: Previous research on the etiology of ADHD symptoms suggests that neuropsychological differences may be present as early as birth; however, the diagnosis is typically not given until school age. This study aimed to (a) identify early behavioral and cognitive markers of later significant parent and/or teacher ratings of ADHD…

Mathematics Recovery (MR) is designed to identify first graders who are struggling in mathematics and provide them with intensive one-to-one tutoring. We report findings from a 2-year evaluation of MR conducted in 20 elementary schools across five districts in two states. The design allowed for the estimation of the counterfactual growth…

This mixed-methods study examines the effects of rating scale length and rater experience on listeners' judgments of second-language (L2) speech. Twenty experienced and 20 novice raters, who were randomly assigned to 5-point or 9-point rating scale conditions, judged speech samples of 38 newcomers to Canada on numerical rating scales for…

Understanding the sources of variance in speaking assessment is important in Japan where society's high demand for English speaking skills is growing. Three challenges threaten fair assessment of speaking. First, in Japanese university speaking courses, teachers are typically the only raters, but teachers' knowledge of their students may unfairly…

This dissertation analyzes instructional leadership and evaluation protocols from a large, diverse district in the Southeastern United States in order to investigate layers of accountability and tensions created when principals are asked to fill the dual roles of both instructional leader and supervisor in a newly implemented teacher evaluation…

Riley's Stuttering Severity Instrument (SSI) is widely used. The manuals allow SSI assessments to be made in different ways (e.g. from digital recordings or whilst listening to speech live). Digital recordings allow segments to be selected and listened to, whereas the entire recording has to be judged when listened to live. Comparison was made…

In the absence of any national quality performance monitoring system (until 2006) 8 of 21 Greek public universities voluntarily participated in the European University Association Institutional Evaluation Programme (EUA-IEP). Empirical data were collected through documents (eight evaluators' reports). This article uses qualitative content analysis…

Continuing professional development (CPD) evaluation in education has been heavily influenced by "level models", deriving from the work of Kirkpatrick and Guskey in particular, which attempt to trace the processes through which CPD interventions achieve outcomes. This paper considers the strengths and limitations of such models, and in…

Although scholars have proposed many steps to increase evaluation use, there has been little comparative empirical study of whether researchers follow these recommendations and whether specific steps are associated with greater utilization. This study of state legislative evaluators finds that those that regularly meet with stakeholders and…