The recent commentary by Beaujean and Farmer (2020) on the original paper by Dixon et al. (2019) serves a cautionary tale of selective p-values, the law of small N sizes, and the type-II error. We believe these authors have crafted a somewhat questionable argument in which only 57% of the original Dixon et al. data were re-analyzed, based on a…

Standard meta-analysis methods are vulnerable to bias from incomplete reporting of results (both publication and outcome reporting bias) and poor study quality. Several alternative methods have been proposed as being less vulnerable to such biases. To evaluate these claims independently we simulated study results under a broad range of conditions…

Although the null hypothesis significance testing procedure is problematic, many still favor the use of "p"-values as indicating the state of evidence against the model used to generate the "p"-value. From this perspective, "p"-values benefit science; or would benefit science if used correctly. In contrast, the novel…

When a psychologist announces a new research finding, it is often based on a rejected null hypothesis. However, if that hypothesis is true, the claim is a false alarm. Many students mistakenly believe that the probability of committing a false alarm equals alpha, the criterion for statistical significance, which is typically set at 5%. Instructors…

In this paper, we investigate two purported problems with Bayesian Knowledge Tracing (BKT), a popular statistical model of student learning: "identifiability" and "semantic model degeneracy." In 2007, Beck and Chang stated that BKT is susceptible to an "identifiability problem"--various models with different…

Recently, Stephen Gorard has outlined strong objections to the use of significance testing in social research. He has argued, first, that as the samples used in social research are almost always non-random it is not possible to use inferential statistical techniques and, second, that even if a truly random sample were achieved, the logic behind…

We present a case study of the process through which a methodology was developed and applied to a quasi-experimental research study that employed propensity score matching. Methodological decisions are discussed and summarized, including an explanation of the approaches selected for each step in the study as well as rationales for these…

Studies of homeless individuals typically sample subjects from few types of sites or regions within a metropolitan area. This article focuses on the biases that can result from such a practice. We obtained a probability sample of 419 homeless youth from 41 sites (shelters, drop-in centers, and streets) in four regions of Los Angeles County (LAC).…

Randomized experiments are commonly used to evaluate if particular interventions improve student achievement. While these experiments can establish that a treatment actually "causes" changes, typically the participants are not randomly selected from a well-defined population and therefore the results do not readily generalize. Three…

This methodological note illustrates how a commonly used calculation of the Delta-p statistic is inappropriate for categorical independent variables, and this note provides users of logistic regression with a revised calculation of the Delta-p statistic that is more meaningful when studying the differences in the predicted probability of an…

Examines and rejects common criticisms of the causality concept; shows causality is a relation implied in the grammar of a language about objects. Discusses objective criteria for concepts of causal relations and explains how the concept of causality may be modified to have causes determine probabilities of outcomes. (Author/RH)

Discusses problems with the use of stochastic (probabilistic) models for the analysis of juvenile offense data. Analyzes longitudinal data for White and non-White delinquents which show significant evidence of offense specialization and a random distribution of offenses if no specialization occurs. (Author/GC)

A novel solution to the statistical problems in an evaluation of rare events is described. The significance of variations in the number of child homicides is analyzed in a binary time series of "active" months for monitoring future incidence and related systemic events. (Author/CM)

Argues that Gottlieb rejects population sampling and statistical analyses of distributions as he proposes that his experimental brand of mechanistic science is the only legitimate approach to developmental research. Maintains that Gottlieb exaggerates developmental uncertainty, based on his own research with extreme environmental manipulations.…