The goal of this study was to explore the effectiveness of a short web-based tutorial in helping teachers to better understand the portrayal of measurement error in test score reports. The short video tutorial included both verbal and graphical representations of measurement error. Results showed a significant difference in comprehension scores…

A problem that is frequently encountered during the systematic review process is when studies that meet the inclusion criteria do not provide the appropriate numerical estimates to include in a meta-analysis. For dichotomous outcomes, a method has been suggested by Di Pietrantonj for reconstructing the 2 × 2 table when the Odds Ratio…

Experimental evaluations that involve the educational system usually involve a hierarchical structure (students are nested within classrooms that are nested within schools, etc.). Concerns about contamination, where research subjects receive certain features of an intervention intended for subjects in a different experimental group, have often led…

We investigated methods of including covariates in two-level models for cluster randomized trials to increase power to detect the treatment effect. We compared multilevel models that included either an observed cluster mean or a latent cluster mean as a covariate, as well as the effect of including Level 1 deviation scores in the model. A Monte…

The paper shows how the methodology for a pedagogical experiment can be improved through including the pre-research stage. If the experiment has the form of a test procedure, an improvement of methodology can be achieved using for example the methods of statistical and didactic analysis of tests which are traditionally used in other areas, i.e.…

Randomized controlled trials (RCTs) are considered the "gold standard" for evaluating an intervention's effectiveness. Recently, the federal government has placed increased emphasis on the use of opportunistic experiments. A key criterion for conducting opportunistic experiments, however, is that there is relatively easy access to data…

Language learners' productive role in teaching and learning processes has recently been the focus of attention. Therefore, this study aimed at investigating the effect of oral vs. written output-based instruction on English as a foreign language (EFL) learners' vocabulary learning with a focus on reflective vs. impulsive learning styles. To this…

When lotteries are infeasible, researchers must rely on observational methods to estimate charter effectiveness at raising student test scores. Considerable attention has been paid to observational studies by the Stanford Center for Research on Education Outcomes (CREDO), which have analyzed charter performance in 27 states. However, the…

This "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)" provides a detailed description of the standards and procedures of the What Works Clearinghouse (WWC). The remaining chapters of this Handbook are organized to take the reader through the basic steps that the WWC uses to develop a review protocol, identify…

The impact of socially desirable responding or faking on noncognitive assessments remains an issue of strong debate. One of the main reasons for the controversy is the lack of a statistical method to model such response sets. This article introduces a new way to model faking based on the assumption that faking occurs due to an interaction between…

For RCTs of education interventions, it is often of interest to estimate associations between student and mediating teacher practice outcomes, to examine the extent to which the study's conceptual model is supported by the data, and to identify specific mediators that are most associated with student learning. This paper develops statistical power…

This paper examines the estimation of two-stage clustered RCT designs in education research using the Neyman causal inference framework that underlies experiments. The key distinction between the considered causal models is whether potential treatment and control group outcomes are considered to be fixed for the study population (the…

Presents an alternative statistical test, BOOT(subscript)med for the two-group situation when a small experimental group is being compared with a large control group. BOOTmed is a between-groups median test derived through bootstrapping techniques. Empirical validation indicates that BOOTmed maintains relatively robust error rates under a variety…

This paper reports the findings of an experiment that investigated the reactions of Saudi college students to collaborative learning techniques introduced in two modalities: face-to-face and web-based learning. Quantitative data were collected with a questionnaire that examined the changes of three constructs: attitudes toward collaboration,…

Action research often necessitates the use of intact groups for the comparison of educational treatments or programs. This paper considers several analytical methods that might be used for such situations when pretest scores indicate that these intact groups differ significantly initially. The methods considered include gain score analysis of…