This paper argues for three aspects of tolerance with respect to QCA research: tolerance with respect to different approaches to QCA; producing QCA research with tolerance (work that is resistant to criticism); and for QCA researchers to be clear about the tolerance of the solutions they present -- especially in terms of calibration and truth…

The role of knowledge and reading comprehension has seen a recent explosion of attention from researchers, journalists, and policy advocates. Much of this discourse describes knowledge in neutral terms such as knowledge of "the world". That knowledge of the world, however, is woven into the fabric of the gendered world we live in and its…

The Campbellian validity typology has been used as a foundation for outcome evaluation and for developing evidence-based interventions for decades. As such, randomized control trials were preferred for outcome evaluation. However, some evaluators disagree with the validity typology's argument that randomized controlled trials as the best design…

Given the widespread use of non-experimental (NE) methods for assessing program impacts, there is a strong need to know whether NE approaches yield causally valid results in field settings. In within-study comparison (WSC) designs, the researcher compares treatment effects from an NE with those obtained from a randomized experiment that shares the…

Mixed methods research (MMR) is a useful paradigm for group work as it allows exploration of both participant outcomes and "how" or "why" such changes occur. Unfortunately, the group counseling literature is not replete with MMR studies. This article reviews the application of MMR to group contexts and summarizes the corpus of…

Evaluation of academic research plays a significant role in government efforts to steer public universities. The scope of such evaluation is now being extended to include the "relevance" or "impact" of academic research outside the academy. We address how evaluation of non-academic research impact can promote more such impact…

There is significant controversy over what counts as evidence in the evaluation of social interventions. It is increasingly common to use methodological criteria to rank evidence types in a hierarchy, with Randomised Controlled Trials (RCTs) at or near the highest level. Because of numerous challenges to a hierarchical approach, this article…

This article presents the analysis of evaluation methods used in a well-designed and comprehensive evaluation effort of a significant Extension program. The evaluation data collection methods were analyzed by questionnaire mode and timing of follow-up surveys. Response rates from the short- and long-term follow-ups and different questionnaire…

The 4-H youth development organization is a complex public--private partnership between the U.S. Department of Agriculture's National Institute of Food and Agriculture, the nation's Cooperative Extension system and National 4-H Council, a private, nonprofit partner. The current article is focused on a partnership approach to the…

Non-randomized studies may provide valuable evidence on the effects of interventions. They are the main source of evidence on the intended effects of some types of interventions and often provide the only evidence about the effects of interventions on long-term outcomes, rare events or adverse effects. Therefore, systematic reviews on the effects…

Critical appraisal of the research literature is an essential step in informing and implementing evidence-based practice. Quality appraisal tools that assess the methodological quality of experimental studies provide a means to identify the most rigorous research suitable for evidence-based decision-making. In single-subject experimental research,…

Although randomized studies have high internal validity, generalizability of the estimated causal effect from randomized clinical trials to real-world clinical or educational practice may be limited. We consider the implication of randomized assignment to treatment, as compared with choice of preferred treatment as it occurs in real-world…

This content analysis examined internationally focused scholarship published from 1997 to 2009 in two counseling journals published in the United States (US), namely "The Counseling Psychologist" ("TCP") and the "Journal of Counseling Psychology" ("JCP"). Both demographic and content criteria, including…

Background/Context: Principles of evidential reasoning have often been discussed in the context of educational and psychological measurement with respect to construct validity and validity arguments. More recently, Mislevy proposed the metaphor of assessment as an evidentiary argument about students' learning and abilities given their…

The success of Response-to-Intervention (RTI) and similar models of service delivery is dependent on educators being able to apply effective and efficient remedial procedures. In the process of implementing problem-solving RTI models, school psychologists have an opportunity to contribute to and enhance the quality of our remedial-procedure…