Experimental studies of educational interventions are rarely based on representative samples of the target population. This simulation study tests two formal sampling strategies for selecting districts and schools from within strata when they may not agree to participate if selected: (1) balanced selection of the most typical district or school…

Large-scale education data are collected via complex sampling designs that incorporate clustering and unequal probability of selection. Multilevel models are often utilized to account for clustering effects. The probability weighted approach (PWA) has been frequently used to deal with the unequal probability of selection. In this study, we examine…

Single subject (SS) designs are popular in educational and psychological research. There exist several statistical techniques designed to analyze such data and to address the question of whether an intervention has the desired impact. Recently, researchers have suggested that generalized additive models (GAMs) might be useful for modeling…

There is growing pressure to make efficacy experiments more useful. This requires attending to the twin goals of generalizing experimental results to those schools that will use the results and testing the intervention's theory of action. We show how electronic records, created naturally during the daily operation of technology-based…

Randomized experiments are considered the gold standard for causal inference because they can provide unbiased estimates of treatment effects for the experimental participants. However, researchers and policymakers are often interested in using a specific experiment to inform decisions about other target populations. In education research,…

In education research, small samples are common because of financial limitations, logistical challenges, or exploratory studies. With small samples, statistical principles on which researchers rely do not hold, leading to trust issues with model estimates and possible replication issues when scaling up. Researchers are generally aware of such…

Despite the increasing popularity of latent class models (LCM) in educational research, methodological studies have not yet accumulated much information on the appropriate application of this modeling technique, especially with regard to requirement on sample size and number of indicators. This dissertation study represented an initial attempt to…

Standard statistical methods are used to analyze data that is assumed to be collected using a simple random sampling scheme. These methods, however, tend to underestimate variance when the data is collected with a cluster design, which is often found in educational survey research. The purposes of this paper are to demonstrate how a cluster design…

The contrasting groups method is one of many possible methods for setting passing scores. The most commonly used method is probably that developed by W. H. Angoff (1971), but it has been suggested that the Angoff method may not be appropriate for many standard setting applications in education. The contrasting groups method is explored as an…

The utility of the biomedical model of adjusted statistics is demonstrated. The model is recommended for use by educational researchers to randomize subjects for a more accurate estimate of school programs' success or failure when compared across classrooms or other units. (Author/MH)

This paper provides an overview of common sampling methods (both the good and the bad) likely to be used in community college self-evaluations and presents the results from several simulated trials. The report begins by reviewing various survey techniques, discussing the negative and positive aspects of each method. The increased accuracy and…