Multisite trials, in which individuals are randomly assigned to alternative treatment arms within sites, offer an excellent opportunity to estimate the cross-site average effect of treatment assignment (intent to treat or ITT) "and" the amount by which this impact varies across sites. Although both of these statistics are substantively…

The paper shows how the methodology for a pedagogical experiment can be improved through including the pre-research stage. If the experiment has the form of a test procedure, an improvement of methodology can be achieved using for example the methods of statistical and didactic analysis of tests which are traditionally used in other areas, i.e.…

We used a false-biofeedback methodology to manipulate physiological arousal in order to induce affective states that would influence learners' metacognitive judgments and learning performance. False-biofeedback is a method used to induce physiological arousal (and resultant affective states) by presenting learners with audio stimuli of false heart…

The purpose of this document is to describe the methods RAND used to analyze achievement for 23 personalized learning (PL) schools for the 2012-13 through 2013-14 academic years. This work was performed at the request of the Bill & Melinda Gates Foundation (BMGF), as part of a multi-year evaluation contract. The 23 schools were selected from a…

The motivation for this paper is the authors' recent work on several randomized control trials in which they found the primary result, which averaged across subgroups or sites, to be moderated by demographic or site characteristics. They are led to examine a distinction that the Institute of Education Sciences (IES) makes between "confirmatory"…

Virtual learning is a popular teaching modality, and it has been reported in research that there is no significant difference in academic outcome as compared with face-to-face courses. Not all researchers agree with this assertion and some claim it is more difficult to teach math-oriented subjects online. Given that educational psychology theories…

Studies that examine the impacts of education interventions on key student, teacher, and school outcomes typically collect data on large samples and on many outcomes. In analyzing these data, researchers typically conduct multiple hypothesis tests to address key impact evaluation questions. Tests are conducted to assess intervention effects for…

One of the most frequently encountered problems in educational research and evaluation is that of evaluating the effect of a treatment in settings over which the researcher or evaluator has little control. A strong plea is made for use of quasi-experimental designs in these situations and this research provides a method of testing the hypothesis…

An approach to the analysis of an aptitude-treatment-interaction (ATI) design in which the treatment groups are based on an underlying factorial structure is described and illustrated. The approach emphasizes description with point and interval estimation. The example design considered consisted of two nominal treatment variables and one interval…

The desirability of combining tests of theory with evaluations of treatment modalities is argued in an investigation of the effectiveness of a juvenile diversion program. Using a true experimental design (with randomization), recidivism analyses dependent on court record data failed to demonstrate the relative superiority of any of three treatment…

The purpose of this study was to develop the Johnson-Neyman Procedure (JN-Procedure) appropriate to multiple groups and covariables, and demonstrate its use in the analysis of group differences. A sequence of significance tests which makes it possible to identify the most parsimonious analysis of group differences appropriate to a given set of…

Presents a prototypical case illustrating what positivist research means and argues that even critics of positivist research are logically committed to propositions that can be tested only through positivist research paradigms. The relationship between the nature of a community's research enterprise and its educational provisions must be causal.…

The purpose of this study was to determine whether more or less comprehension occurred on the part of the human receiver when information was transmitted in a verbalized, compressed, repetitive manner through the auditory channel in the same unit of time necessary for one "normal" transmission. A research hypothesis was developed which…

In order to make policy decisions, educators must evaluate educational research proposals and projects. Findings immediately related to practice are often inadequate, omitting the theoretical establishment of principles by which we can explain and predict the phenomena of our world. These theoretical linkages to the practical world of…