It is common to include multiple dependent variables (DVs) in single-case experimental design (SCED) meta-analyses. However, statistical issues associated with multiple DVs in the multilevel modeling approach (i.e., possible dependency of error, heterogeneous treatment effects, and heterogeneous error structures) have not been fully investigated.…

The performance of statistical methods is often evaluated by means of simulation studies. In case of network meta-analysis of binary data, however, simulations are not currently available for many practically relevant settings. We perform a simulation study for sparse networks of trials under between-trial heterogeneity and including multi-arm…

We study the relative performance of two policy-relevant value-added models--a one-step fixed effect model and a two-step aggregated residuals model--using a simulated data set well grounded in the value-added literature. A key feature of our data generating process is that student achievement depends on a continuous measure of economic…

In this paper, I extend the concept of observer effect into the realm of country-level secondary data analysis. When analyzing what appear to be the same secondary data using the same methods, macro-comparative researchers arrive at different results. I argue that this is a product of idiosyncratic variation directly or indirectly produced by the…

Recently Watkins, Bargagliotti, and Franklin (2014) discovered that simulations of the sampling distribution of the mean can mislead students into concluding that the mean of the sampling distribution of the mean depends on sample size. This potential error arises from the fact that the mean of a simulated sampling distribution will tend to be…

This article discusses the nonuse, misuse, and proper use of pilot studies in experimental evaluation research. The authors first show that there is little theoretical, practical, or empirical guidance available to researchers who seek to incorporate pilot studies into experimental evaluation research designs. The authors then discuss how pilot…