In their essay, "Why Assessment Will Never Work...," Bacon and Stewart (2016) recommend that instead of carrying out the expensive process of experimenting themselves, many business schools would get a bigger bang for their buck if they used "published pedagogical studies that use direct measures of learning with sufficient…

Assessing societal impacts of research is more difficult than assessing advances in knowledge. Methods to evaluate research impact on policy processes and outcomes are especially underdeveloped, and are needed to optimize the influence of research on policy for addressing complex issues such as chronic diseases. Contribution analysis (CA), a…

On the long and arduous journey toward effective educational assessment, business schools have progressed in their ability to clearly state measurable learning goals and use direct measures of student learning. However, many schools are wrestling with the last stages of the journey--measuring present learning outcomes, implementing…

The objective of this commentary is to call attention to the feasibility and importance of large-scale, systematic, quantitative analysis in international and comparative education research. We contend that although many existing databases are under- or unutilized in quantitative international-comparative research, these resources present the…

The essay examines the variety of research methods and measures used in the first-year experience and students-in-transition field over the past 25 years. Interrogating the extant research, Kinzie explores whether the methods and analytic processes most commonly employed are adequate to advance our understanding of complex issues in the field. The…

No Child Left Behind (NCLB) and the Individuals with Disabilities Education Act (IDEA) have increased the emphasis on analyzing data on disproportionality. Data required to answer basic questions include: (1) total number of students enrolled; (2) total number of each racial/ethnic group enrolled; (3) total number of students in the category under…

Methodological and political issues arise during the designing, conducting, and reporting of campus-climate studies for LGBT students. These issues interact; making a decision about a methodological issue (e.g., sample size) has an impact on a political issue (e.g., how well the findings will be received). Ten key questions that must be addressed…

Over the past few years Research Data Services (RDS), part of the National Foundation for Educational Research (NFER), has led numerous large-scale, longitudinal projects. Feedback to schools has been provided for all of these projects, and has been in the form of sets of tables and charts and electronic files illustrating pupil performance and…

Numerous misuses of statistics are described and illustrated, and ways of recognizing and avoiding such misuse are discussed. The following five categories of statistical misuse are identified: a lack of knowledge of the subject matter, the quality of the basic data, the preparation of the study and the report, the statistical methodology, and a…

This paper reviews criticism on misinterpretation and overuse of significance testing in the social sciences and examines use of effect size measures to enhance interpretation of significance testing. Review of typical effect size measures and their application is followed by analysis of use of effect size measures in studies reported over 10…

This commentary documents errors in the statistical analyses and interpretation of empirical evidence in "The Bell Curve" by R. Herrnstein and C. Murray (1994). "The Bell Curve" neglects the role of education and grossly overstates the case for IQ as a determinant of social success. (SLD)

In the series introduction, Roblyer (2005) presents a conceptual framework for improving typical research on educational technology. This commentary is intended to extend and deepen Roblyer's very brief discussion of one particular type of research--studies of technology-based instructional designs--that she describes as "almost…

The current state of usage of regression models in analysis of variance (ANOVA) designs is empirically examined, and examples of several statistical errors made in usage are presented. The assumptions of the general linear model are that all predictors are known without error of measurement and are fixed with no replication or sample variation; in…

Using the 1995 National Research Council ratings of doctoral programs in 41 fields, a statistical methodology is offered for assessing how measures of program size, faculty seniority, faculty research productivity, and faculty productivity in producing doctoral degrees influenced raters' decisions. Then using data for one field, economics,…

A careful review of research suggests that the problem of sexual assault has been magnified and the data misinterpreted, in part because the issue of sexual violence against women has become enmeshed in politics. The current trend in sexual-violence research is to redefine it according to the ideological preferences of advocates. (SLD)