Estimating confidence intervals (CIs) for the correlation has been a challenge because the correlation sampling distribution changes depending on the correlation magnitude. The Fisher z-transformation was one of the first attempts at estimating correlation CIs but has historically shown to not have acceptable coverage probability if data were…

The recent commentary by Beaujean and Farmer (2020) on the original paper by Dixon et al. (2019) serves a cautionary tale of selective p-values, the law of small N sizes, and the type-II error. We believe these authors have crafted a somewhat questionable argument in which only 57% of the original Dixon et al. data were re-analyzed, based on a…

Purpose: Course evaluations are formative and are used to evaluate learnings of the students for a course. Anomalies in the evaluation process can lead to a faulty educational outcome. Learning analytics and educational data mining provide a set of techniques that can be conveniently applied to extensive data collected as part of the evaluation…

Standard meta-analysis methods are vulnerable to bias from incomplete reporting of results (both publication and outcome reporting bias) and poor study quality. Several alternative methods have been proposed as being less vulnerable to such biases. To evaluate these claims independently we simulated study results under a broad range of conditions…

The standard regression technique for modeling binary response variables in education research is logistic regression. The odds ratios from these models are used to quantify and communicate variable effects. These effects are sometimes pooled together as in a meta-analysis. We argue that this process is problematic as odds ratios calculated from…

Although the null hypothesis significance testing procedure is problematic, many still favor the use of "p"-values as indicating the state of evidence against the model used to generate the "p"-value. From this perspective, "p"-values benefit science; or would benefit science if used correctly. In contrast, the novel…

The purpose of this study was to compare and contrast frameworks for evaluating methodological rigor in single case research. Specifically, research on high-probability requests to increase compliance in young children was evaluated. Ten studies were identified and were coded using 4 frameworks. These frameworks were the Council for Exceptional…

Researchers are combining self-reports from mobile surveys with passive data collection using sensors and apps on smartphones increasingly more often. While smartphones are commonly used in some groups of individuals, smartphone penetration is significantly lower in other groups. In addition, different operating systems (OSs) limit how mobile data…

When a psychologist announces a new research finding, it is often based on a rejected null hypothesis. However, if that hypothesis is true, the claim is a false alarm. Many students mistakenly believe that the probability of committing a false alarm equals alpha, the criterion for statistical significance, which is typically set at 5%. Instructors…

Case-mix heterogeneity across studies complicates meta-analyses. As a result of this, treatments that are equally effective on patient subgroups may appear to have different effectiveness on patient populations with different case mix. It is therefore important that meta-analyses be explicit for what patient population they describe the treatment…

This study shows that residual variation can cause problems related to scaling in exponential random graph models (ERGM). Residual variation is likely to exist when there are unmeasured variables in a model--even those uncorrelated with other predictors--or when the logistic form of the model is inappropriate. As a consequence, coefficients cannot…

There has been much controversy over the null hypothesis significance testing procedure, with much of the criticism centered on the problem of inverse inference. Specifically, p gives the probability of the finding (or one more extreme) given the null hypothesis, whereas the null hypothesis significance testing procedure involves drawing a…

Although social scientists devote considerable effort to mitigating measurement error during data collection, they often ignore the issue during data analysis. And although many statistical methods have been proposed for reducing measurement error-induced biases, few have been widely used because of implausible assumptions, high levels of model…

Recently, Stephen Gorard has outlined strong objections to the use of significance testing in social research. He has argued, first, that as the samples used in social research are almost always non-random it is not possible to use inferential statistical techniques and, second, that even if a truly random sample were achieved, the logic behind…

The effective evaluation of correctional programs is critically important. However, research in corrections rarely allows for the randomization of offenders to conditions of the study. This limitation compromises internal validity, and thus, causal conclusions can rarely be drawn. Increasingly, researchers are employing propensity score matching…