Conventional random-effects models in meta-analysis rely on large sample approximations instead of exact small sample results. While random-effects methods produce efficient estimates and confidence intervals for the summary effect have correct coverage when the number of studies is sufficiently large, we demonstrate that conventional methods…

Checking for possible inconsistency between direct and indirect evidence is an important task in network meta-analysis. Recently, an evidence-splitting (ES) model has been proposed, that allows separating direct and indirect evidence in a network and hence assessing inconsistency. A salient feature of this model is that the variance for…

The partial correlation coefficient (PCC) is used to quantify the linear relationship between two variables while taking into account/controlling for other variables. Researchers frequently synthesize PCCs in a meta-analysis, but two of the assumptions of the common equal-effect and random-effects meta-analysis model are by definition violated.…

Meta-analyses in orphan diseases and small populations generally face particular problems, including small numbers of studies, small study sizes and heterogeneity of results. However, the heterogeneity is difficult to estimate if only very few studies are included. Motivated by a systematic review in immunosuppression following liver…

People with autism show superior performance to controls on the Embedded Figures Test (EFT). However, studies examining the relationship between autistic-like traits and EFT performance in neurotypical individuals have yielded inconsistent findings. To examine the inconsistency, a meta-analysis was conducted of studies that (a) compared high and…

Despite the widespread interest in using and researching simulation in higher education, little discussion has yet to address a key pedagogical concern: difficulty. A "sociomaterial" view of learning, explained in this paper, goes beyond cognitive considerations to highlight dimensions of material, situational, representational and…

In 2 Monte Carlo studies of fixed- and random-effects meta-analysis for correlations, A. P. Field (2001) ostensibly evaluated Hedges-Olkin-Vevea Fisher-[zeta] and Schmidt-Hunter Pearson-r estimators and tests in 120 conditions. Some authors have cited those results as evidence not to meta-analyze Fisher-[zeta] correlations, especially with…

This article examines the state of assessment in simulation and gaming over the past 40 years. While assessment has come slowly to many disciplines, members of the simulation and gaming community have been assessing the educational effectiveness of their experiential activities for years, in part because of skepticism from more traditional…

The Personality Assessment Inventory (L. C. Morey, 1991) includes 3 measures for identifying overreporting of psychopathology: the Negative Impression scale (NIM), Malingering Index (MAL), and Rogers Discriminant Function (RDF). Meta-analysis revealed that each measure was a strong predictor of uncoached (NIM, d = 1.48, k = 23; MAL, d = 1.15, k =…

Federal standards established in 1997 allow respondents to select multiple-race categories. These new standards changed the single-race subgroup definitions that the government has required since 1977. Meta-analysis, research on long-term assessment trends, and other research involving historical comparisons must account for the definitional…

The homogeneity test provided by L. Hedges (1982) in meta-analysis has been widely used, mainly to test if the effect sizes share the same variance. Ignoring the intercorrelations among effect sizes affects the Type I error rate. The main purpose of this research was to study the impact of pooling effect sizes on the homogeneity test in effect…

Practical meta-analysis of correlation matrices generally ignores covariances (and hence correlations) between correlation estimates. The authors consider various methods for allowing for covariances, including generalized least squares, maximum marginal likelihood, and Bayesian approaches, illustrated using a 6-dimensional response in a series of…

Blanton and Jaccard questioned the 4-test regression method used by Greenwald et al. to test a pure multiplicative theory. The present authors address Blanton and Jaccard's concerns with a combination of simulations and meta-analysis. Simulations show that (a) Blanton and Jaccard's preferred simultaneous regression method has a severe power loss…

Studied the effect of including nonindependent correlations in the meta-analysis method of J. Hunter and F. Schmidt on the estimated population standard deviation. Evaluation indicates that the Hunter and Schmidt method will underestimate the true population standard deviation. Developed new methods to correct for this and illustrated the methods…

Using a set of scenarios derived from actual evaluation studies, this simulation study examines the reported influence of evaluation information on decision makers' potential actions. Each scenario described a context where one of three types of evaluation information (large-scale study data, case study data, or anecdotal accounts) is presented…