Since 2007, two taxonomies have been proposed to identify the components of evaluation practice that may be specified in an evaluation policy. Little is known, however, about how these taxonomies align with evaluation policies developed by philanthropic foundations. Through thematic analysis, this article first compares 12 foundation evaluation…

The reason for conducting the present case study was the problems the researchers encountered during data collection for another research project (Primary Study) entitled "The effects of virtual versus traditional flipped classes on EFL learners' grammar knowledge, self-regulation, and autonomy." Two online questionnaires were…

Assessment rubrics are being promoted and introduced into tertiary teaching practices on the grounds that they are an efficient and reliable tool to evaluate student performance effectively and promote student learning. However, there has been little discussion on the value of using assessment rubrics in higher education. Rather, they are being…

Cost-effectiveness analysis is rarely used in education. When it is used, it often fails to meet methodological standards, especially with regard to cost measurement. Although there are occasional criticisms of these failings, we believe that it is useful to provide a listing of the more common concerns and how they might be addressed. Based upon…

Despite considerable interest in the topic of instructional quality in research as well as practice, little is known about the quality of its assessment. Using generalizability analysis as well as content analysis, the present study investigates how reliably and validly instructional quality is measured by observer ratings. Twelve trained raters…

In their article Ruscio et al. (Ruscio, Seaman, D'Oriano, Stremlo, & Mahalchik, this issue) present a comparative study of some of the different variants of the "h" index. The study evaluates a total of 22 metrics, including the "h" index and "h"-type indices, as well as other conventional measures. The novelty of their work is to a large extent…

Ruscio and colleagues (Ruscio, Seaman, D'Oriano, Stremlo, & Mahalchik, this issue) provide a thoughtful empirical analysis of 22 different measures of individual scholarly impact. The simplest metric is number of publications, which Simonton (1997) found to be a reasonable predictor of career trajectories. Although the assessment of the scholarly…

Ruscio et al. (Ruscio, Seaman, D'Oriano, Stremlo, & Mahalchik, this issue) write of a thing with which scientists and scholars are all too familiar, the assessment of published research and of its authors. The author was startled to discover how little the agenda of the paper seems to engage with factors one relies on for salary and promotion…

Lecoutre, Lecoutre, and Poitevineau (2010) have provided sophisticated grounding for "p[subscript rep]." Computing it precisely appears, fortunately, no more difficult than doing so approximately. Their analysis will help move predictive inference into the mainstream. Iverson, Wagenmakers, and Lee (2010) have also validated…

Ruscio, Seaman, D'Oriano, Stremlo, and Mahalchik (this issue) evaluate 22 bibliometric indicators, including conventional measures, like the number of publications, the "h" index, and many "h" index variants. To assess the quality of the indicators, their well-justified criteria encompass conceptual, empirical, and practical…

P. R. Killeen's (2005a) probability of replication ("p[subscript rep]") of an experimental result is the fiducial Bayesian predictive probability of finding a same-sign effect in a replication of an experiment. "p[subscript rep]" is now routinely reported in "Psychological Science" and has also begun to appear in…

The sense that replicability is an important aspect of empirical science led Killeen (2005a) to define "p[subscript rep]," the probability that a replication will result in an outcome in the same direction as that found in a current experiment. Since then, several authors have praised and criticized 'p[subscript rep]," culminating…

This comment offers three descriptions of "p[subscript rep]" that start with a frequentist account of confidence intervals, draw on R. A. Fisher's fiducial argument, and do not make Bayesian assumptions. Links are described among "p[subscript rep]," "p" values, and the probability a confidence interval will capture…

In this commentary, the authors discuss some of the issues regarding the use of diagnostic classification models that practitioners should keep in mind. In the authors experience, these issues are not as well known as they should be. The authors then provide recommendations on diagnostic scoring.

This paper forms part of an exploration of assessment on one part-time higher education (HE) course: an in-service, professional qualification for teachers and trainers in the learning and skills sector which is delivered on a franchise basis across a network of further education colleges in the north of England. This paper proposes that the…