We describe best practices in reporting reliability estimates in counseling research with consideration to precision, generalization, and diverse populations. We provide a historical context to reporting reliability estimates, the limitations of past practices, and new methods to address reliability generalization. We highlight best practices…

To enable optimal decision-making based on the best evidence available, open syntheses are called for. To make data accessible and comprehensible even for decision-makers without proficient knowledge in meta-analysis, a graphical user interface (GUI) provides flexible data visualizations including interpretation aids. Moreover, due to a growing…

Meta-analyses can be used to comprehensively summarize the state of a given literature base, understand development and relations between constructs, and synthesize intervention effects to identify "what works for whom," all of which can directly inform research, practice, and policy. In this tutorial, we first argue that data reporting…

Researchers often seek to synthesize results of multiple studies on the same topic to draw statistical or substantive conclusions and to estimate effect sizes that will inform power analyses for future research. The most popular synthesis approach is meta-analysis. There have been few discussions and applications of other synthesis approaches.…

Longitudinal data are common and essential to understanding human development. This paper introduces an approach to synthesizing longitudinal research findings called lag as moderator meta-analysis (LAMMA). This approach capitalizes on between-study variability in time lags studied in order to identify the impact of lag on estimates of stability…

High-quality systematic literature reviews provide a systematic process for identifying, synthesizing, and critiquing multiple studies and, in turn, inform theory, research, practice, and policy. With a focus on special education systematic reviews, we propose four core principles (i.e., coherence, contextualization, generativity, and…

Extensive debate of potentially common, yet questionable research practices that lead to biased findings within social and health sciences has emerged over the last decade. These challenges likely apply to educational psychology, though the field has been slow to address them. This article discusses current research norms, strategic solutions…

For a variety of reasons, education research can be difficult to summarize. Varying contexts, designs, levels of quality, measurement challenges, definition of underlying constructs, and treatments as well as the complexity of research subjects themselves can result in variability. Education research is voluminous and draws on multiple methods…

In systematic reviews, meta-analyses are routinely applied to summarize the results of the relevant studies for a specific research question. If one can assume that in all studies the same true effect is estimated, the application of a meta-analysis with common effect (commonly referred to as fixed-effect meta-analysis) is adequate. If…

Network meta-analysis enables the simultaneous synthesis of a network of clinical trials comparing any number of treatments. Potential inconsistencies between estimates of relative treatment effects are an important concern, and several methods to detect inconsistency have been proposed. This paper is concerned with the node-splitting approach,…

Latino males across the country enroll in community colleges with the purpose of obtaining an educational degree, which could lead to accomplishing professional and personal aspirations. Even if Latino male students enroll in post-secondary education, they continue to be disenfranchised, vanished, and often rejected through the higher education…

In this article, we trace the development of the Campbell Collaboration and its renewed efforts to build a world library of accurate, synthesized evidence to inform policy and practice and improve human well-being worldwide. Campbell systematic reviews and related evidence synthesis products provide unbiased summaries of entire bodies of empirical…

The authors provide a template for counseling researchers who are interested in quantitatively aggregating research findings. Meta-analytic studies can provide relevant information to the counseling field by systematically synthesizing studies performed by researchers from diverse fields. Methodologically sound meta-analyses require careful…

Research synthesis of evaluation findings is a multistep process. An investigator identifies a research question, acquires the relevant literature, codes findings from that literature, and analyzes the coded data to estimate the average treatment effect and its distribution in a population of interest. The process of estimating the average…

Systematic reviews and meta-analysis are techniques for synthesizing primary empirical studies to produce a summary of effects. To facilitate this goal, the Campbell Collaboration (C2) supports reviews within the disciplines of crime and justice, education, international development, and social welfare. At the annual Campbell Colloquium, experts…