As evaluators, we are often asked to determine whether policies and programs provide value for the resources invested. Addressing that question can be a quandary, and, in some cases, evaluators question whether cost-benefit analysis is fit for this purpose. With increased interest globally in social enterprise, impact investing, and social impact…

The article serves as a guide for researchers in developing evidence of validity using bivariate correlations, specifically construct validity. The authors outline the steps for calculating and interpreting bivariate correlations. Additionally, they provide an illustrative example and discuss the implications.

The increasing number of publications on recommender systems for Technology Enhanced Learning (TEL) evidence a growing interest in their development and deployment. In order to support learning, recommender systems for TEL need to consider specific requirements, which differ from the requirements for recommender systems in other domains like…

The literature on bibliometric indices for assessing scholarly impact, in particular the "h" index (Hirsch, 2005) and its many variants, is extensive, but nevertheless Ruscio and colleagues (this issue) succeed in making a valuable contribution. They have made the effort of collecting publication and citation data for no less than 1,750…

When lotteries are infeasible, researchers must rely on observational methods to estimate charter effectiveness at raising student test scores. Considerable attention has been paid to observational studies by the Stanford Center for Research on Education Outcomes (CREDO), which have analyzed charter performance in 27 states. However, the…

Ruscio and colleagues (Ruscio, Seaman, D'Oriano, Stremlo, & Mahalchik, this issue) have done a great service by systematically comparing indices of scholarly impact. Three aspects of their work are particularly valuable: (1) Their assessment of the proliferating collection of metrics, whose development has become something of a cottage industry,…

In their article Ruscio et al. (Ruscio, Seaman, D'Oriano, Stremlo, & Mahalchik, this issue) present a comparative study of some of the different variants of the "h" index. The study evaluates a total of 22 metrics, including the "h" index and "h"-type indices, as well as other conventional measures. The novelty of their work is to a large extent…

Describing, explaining, and discussing various modern indices of scholarly impact as accomplished by Ruscio, Seaman, D'Oriano, Stremlo, and Mahalchik (this issue) is highly commendable, as such measures get increasingly important in hiring and promotion decisions. The author agrees with almost all points made in the target article, except the…

This essay is a contribution to the literature of critique of higher education policy and management. It offers a general discussion of problems arising from the proliferation of mechanisms of audit and quality assurance within the context of concerns about dependence on quantitative measures of values. It argues that uncritical dependence on…

Assessment of higher order learning outcomes such as critical thinking, problem solving and creativity has remained a challenge for universities. While newer technologies such as social networking tools have the potential to support these intended outcomes, academics' assessment practice is slow to change. University mission statements and unit…

This study explores No Child Left Behind's required timetable for English language learners (ELLs) to reach English language proficiency within five years, as outlined in the Annual Measurable Achievement Outcomes (AMAOs), despite the lack of research evidence to support this as a reasonable expectation. Analysis was conducted on the archived data…

In 2002, voters in Florida approved a constitutional amendment limiting class sizes in public schools to 18 students in the elementary grades, 22 students in middle grades, and 25 in high school grades. Analyzing statewide achievement data for school districts from 2004-2006 and for schools in 2007, this study purports to find that "mandated…

Despite the best of intentions, qualitative researchers can be faced, in some circumstances, with having to make meaning from thin, or less than optimal, data. Using a real study as context, the authors describe the ways that they made sense of their thin data on teachers' perceptions of a large-scale evaluation instrument. They propose a…

This article examines the statistical correction for attenuation and the controversies surrounding the procedure. The logic of the double and single correction formulas is discussed, and misapplications and misinterpretations of the correction are presented. (SLD)

Evaluation research is discussed as an applied social science that uses quantitative-comparative techniques to assess the effects of planned intervention. Focus is on the generally insufficient validity of treatment effectiveness research, and the relationship between research constructs and theory and their operationalization. (TJH)