I discuss a published paper in political science that made a claim that aroused skepticism. The reanalysis is an example of how we, as consumers as well as producers of science, can engage with published work. This can be viewed as a sort of collaboration performed implicitly between the authors of a published paper and later researchers who want…

Formal empirical assessments of replication have recently become more prominent in several areas of science, including psychology. These assessments have used different statistical approaches to determine if a finding has been replicated. The purpose of this article is to provide several alternative conceptual frameworks that lead to different…

In their essay, "Why Assessment Will Never Work...," Bacon and Stewart (2016) recommend that instead of carrying out the expensive process of experimenting themselves, many business schools would get a bigger bang for their buck if they used "published pedagogical studies that use direct measures of learning with sufficient…

Structured means analysis is a very useful approach for testing hypotheses about population means on latent constructs. In such models, a z test is most commonly used for testing the statistical significance of the relevant parameter estimates or of the differences between parameter estimates, where a z value is computed based on the asymptotic…

This letter documents some problems in Ancaiani et al. (2015). Namely the evaluation of concordance, based on Cohen's kappa, reported by Ancaiani et al. was not computed on the whole random sample of 9,199 articles, but on a subset of 7,597 articles. The kappas relative to the whole random sample were in the range 0.07-0.15, indicating an…

Applied researchers have employed chi-square tests for more than one hundred years. This paper addresses the question of how one should follow a statistically significant chi-square test result in order to determine the source of that result. Four approaches were evaluated: calculating residuals, comparing cells, ransacking, and partitioning. Data…

Quantitative results from empirical studies are common in the field of Scholarship of Teaching and Learning (SoTL), but it is important to remain aware of what the results from our studies can, and cannot, tell us. Oftentimes studies conducted to examine teaching and learning are constrained by class size. Small sample sizes negatively influence…

Traditions of statistical significance testing in second language (L2) quantitative research are strongly entrenched in how researchers design studies, select analyses, and interpret results. However, statistical significance tests using "p" values are commonly misinterpreted by researchers, reviewers, readers, and others, leading to…

Recently, Stephen Gorard has outlined strong objections to the use of significance testing in social research. He has argued, first, that as the samples used in social research are almost always non-random it is not possible to use inferential statistical techniques and, second, that even if a truly random sample were achieved, the logic behind…

Various studies show that higher education institutions contribute to regional economic development by R&D, creation of human capital, knowledge and technology transfer, and by creation of a favourable milieu. It is brought out that the basic procedure is to sum expenditures of the college community (students, faculty, staff and visitors)…

In this report, a school choice advocacy group presents results from its survey of K-12 parents within and across the public and private sectors. They report that parents are highly satisfied with voucher and tax credit scholarship programs and suggest that the findings support the expansion of school choice programs. However, these and other…

The purpose of this article was to present a synthesis of the peer-reviewed differential bundle functioning (DBF) research that has been conducted to date. A total of 16 studies were synthesized according to the following characteristics: tests used and learner groups, organizing principles used for developing bundles, DBF detection methods used,…

I discuss three common practices that obfuscate or invalidate the statistical analysis of randomized controlled interventions in applied linguistics. These are (a) checking whether randomization produced groups that are balanced on a number of possibly relevant covariates, (b) using repeated measures ANOVA to analyze pretest-posttest designs, and…

The focus of this article is two-fold. First, it describes a model that can be used to guide the evaluation of teacher professional development. The model combines important components of existing models and incorporates the use of students' perceptions for examining teacher change. Second, the article reports the evaluation of a teacher…

This study examined the extent to which school outdoor activities could enhance senior secondary (SS) two students' achievement in ecology. Non randomized pre test post test control group Quasi-experimental design was adopted. A sample of 160 SS II students from 4 co-educational schools in Jalingo metropolis, Taraba State Nigeria was used. A 40…