The purpose of this study was to compare and contrast frameworks for evaluating methodological rigor in single case research. Specifically, research on high-probability requests to increase compliance in young children was evaluated. Ten studies were identified and were coded using 4 frameworks. These frameworks were the Council for Exceptional…

Bayesian statistical approaches to mixed treatment comparisons (MTCs) are becoming more popular because of their flexibility and interpretability. Many randomized clinical trials report multiple outcomes with possible inherent correlations. Moreover, MTC data are typically sparse (although richer than standard meta-analysis, comparing only two…

Despite recent emphasis on the use of randomized control trials (RCTs) for evaluating education interventions, in most areas of education research, observational methods remain the dominant approach for assessing program effects. Over the last three decades, the within-study comparison (WSC) design has emerged as a method for evaluating the…

When performing a meta-analysis unexplained variation above that predicted by within study variation is usually modeled by a random effect. However, in some cases, this is not sufficient to explain all the variation because of outlier or unusual studies. A previously described method is to define an outlier as a study requiring a higher random…

The goal of this research is to make sensitivity analysis accessible not only to empirical researchers but also to the various stakeholders for whom educational evaluations are conducted. To do this it derives anchors for the omitted variable (OV)-program participation association intrinsically, using the Love plot to present a wide range of…

This methodological review draws attention to the challenges faced by intellectual and developmental disabilities researchers in the appropriate design and analysis of group comparison studies. We provide a brief overview of matching methodologies in the field, emphasizing group-matching designs used in behavioral research on cognition and…

Formal inference, which makes theoretical assumptions about distributions and applies hypothesis testing procedures with null and alternative hypotheses, is notoriously difficult for tertiary students to master. The debate about whether this content should appear in Years 11 and 12 of the "Australian Curriculum: Mathematics" has gone on…

The purpose of this study is through Monte Carlo simulation to compare several propensity score methods in approximating factorial experimental design and identify best approaches in reducing bias and mean square error of parameter estimates of the main and interaction effects of two factors. Previous studies focused more on unbiased estimates of…

Propensity score matching (PSM) has become a popular approach for research studies when randomization is infeasible. However, there are significant differences in the effectiveness of selection bias reduction among the existing PSM methods and, therefore, it is challenging for researchers to select an appropriate matching method. This current…

Propensity score methods allow investigators to estimate causal treatment effects using observational or nonrandomized data. In this article we provide a practical illustration of the appropriate steps in conducting propensity score analyses. For illustrative purposes, we use a sample of current smokers who were discharged alive after being…

A well-designed experiment is the best method for establishing efficacy of any intervention, be it medical, behavioral, or educational in nature. This paper reviews the steps necessary in conducting a comparative experiment in an educational setting, and illustrates how these steps might be fulfilled within the context of a large-scale randomized…

This article presents a discussion of mixed methods (MM) sampling techniques. MM sampling involves combining well-established qualitative and quantitative techniques in creative ways to answer research questions posed by MM research designs. Several issues germane to MM sampling are presented including the differences between probability and…

The stability of standard score and probability method sociometric group assignments was examined over a two-year period with an initial group of 334 fifth graders. Popular, neglected, and controversial groups evidenced low stability of group members over intervals of approximately 1, 6, 12, 18, and 24 months. (Author/RH)

The relative power of the Mann-Whitney statistic, the t-statistic, the median test, a test based on exceedances (A,B), and two special cases of (A,B) the Tukey quick test and the revised Tukey quick test, was investigated via a Monte Carlo experiment. These procedures were compared across four population probability models: uniform, beta, normal,…

The comments presented here were made after the presentation of four papers and commentary by two other symposium participants. They address issues in language comparison and classification. First, comments are made on the papers ("An African Test Case in Comparative Methodology,""The Mathematics of Multilateral Comparison,""Testing a Basic…