Background: The generalizability of a study refers to the extent to which the results and inferences from a sample apply to individuals in a larger target population of inference (Shadish et al., 2002). In practice, the strongest tool to facilitate generalizations is random or probability sampling, which is rare in educational studies (Olsen et…

Background: Multisite designs, also known as blocked designs, are experimental designs in which the random assignment of treatment and control conditions is within each site (or block) after the random selection of sites (or blocks). Multisite designs exhibit remarkable adaptability and, statistically, it can maintain a rigorous basis for…

Parallel analysis has been considered one of the most accurate methods for determining the number of factors in factor analysis. One major advantage of parallel analysis over traditional factor retention methods (e.g., Kaiser's rule) is that it addresses the sampling variability of eigenvalues obtained from the identity matrix, representing the…

Prior research has suggested that clustered regression discontinuity designs are a formidable alternative to cluster randomized designs because they provide targeted treatment assignment while maintaining a high-quality basis for inferences on local treatment effects. However, methods for the design and analysis of clustered regression…

This technical report provides detailed information about the sample, survey instruments, and resultant data for the Fall 2024 Pre-Kindergarten Teacher Survey (PKTS) which was administered via RAND's American Teacher Panel (ATP). The ATP is a nationally representative sample of public school teachers and part of the broader American Educator…

When performing an aggregate data meta-analysis of a continuous outcome, researchers often come across primary studies that report the sample median of the outcome. However, standard meta-analytic methods typically cannot be directly applied in this setting. In recent years, there has been substantial development in statistical methods to…

A simulation study was conducted to examine the efficacy of nine frequently-used HOV tests, including Levene's tests with squared residuals and with absolute residuals, Brown and Forsythe (BF) test, Bootstrap BF test, O'Brien test, Z-variance test, Box-Scheffé (BS) test, Bartlett test, and Pseudo jackknife test under comprehensive simulation…

One of the most important developments in the current era of social sciences is the growing availability and diversity of data, big and small. Social scientists increasingly combine information from multiple data sets in their research. While conducting statistical analyses with linked data is relatively straightforward, borrowing information…

While testlets have proven useful for assessing complex skills, the stem shared by multiple items often induces correlations between responses, leading to violations of local independence (LI), which can result in biased parameter and ability estimates. Diagnostic procedures for detecting testlet effects typically involve model comparisons testing…

Recent metascience suggests that motor behavior research may be underpowered, on average. Researchers can perform a priori power analyses to ensure adequately powered studies. However, there are common pitfalls that can result in underestimating the required sample size for a given design and effect size of interest. Critical evaluation of power…

Two- and three-level designs in educational and psychological research can involve entire populations of Level-3 and possibly Level-2 units, such as schools and educational districts nested within a given state, or neighborhoods and counties in a state. Such a design is of increasing relevance in empirical research owing to the growing popularity…

Quantitative evidence synthesis methods aim to combine data from multiple medical trials to infer relative effects of different interventions. A challenge arises when trials report continuous outcomes on different measurement scales. To include all evidence in one coherent analysis, we require methods to "map" the outcomes onto a single…

For ease of instruction in the classroom, the one-way analysis of variance F statistic is rewritten in terms of pairwise differences in individual sample means instead of differences of individual sample means from the overall sample mean. Likewise, the Kruskal-Wallis statistic may be rewritten in terms of pairwise differences in individual…

Education policy research aimed at eliminating racism necessitates methodological innovation that fosters both equity-centered approaches and robust empirical analysis of the systemic nature of racism. Most quantitative research in educational psychology omits the racist environment that students in K-12 education exist in (DeCuir-Gunby &…

Optimal design of multisite randomized trials leverages sampling costs to optimize sampling ratios and ultimately identify more efficient and powerful designs. Past implementations of the optimal design framework have assumed that costs of sampling units are equal across treatment conditions. In this study, we developed a more flexible optimal…