Prior research has suggested that clustered regression discontinuity designs are a formidable alternative to cluster randomized designs because they provide targeted treatment assignment while maintaining a high-quality basis for inferences on local treatment effects. However, methods for the design and analysis of clustered regression…

Two- and three-level designs in educational and psychological research can involve entire populations of Level-3 and possibly Level-2 units, such as schools and educational districts nested within a given state, or neighborhoods and counties in a state. Such a design is of increasing relevance in empirical research owing to the growing popularity…

The purpose of this dissertation is to develop principles and strategies for and identify limitations of multisite cluster randomized trials in the context of partially and fully nested designs. In the first study, I develop principles of estimation, sampling variability, and inference for studies that leverage multisite designs within the context…

Physics education researchers are interested in using the tools of machine learning and natural language processing to make quantitative claims from natural language and text data, such as open-ended responses to survey questions. The aspiration is that this form of machine coding may be more efficient and consistent than human coding, allowing…

Meta-studies are often conducted on empirical findings obtained from overlapping samples. Sample overlap is common in research fields that strongly rely on aggregated observational data (eg, economics and finance), where the same set of data may be used in several studies. More generally, sample overlap tends to occur whenever multiple estimates…

In meta-analysis, effect sizes often need to be converted into a common metric. For this purpose conversion formulas have been constructed; some are exact, others are approximations whose accuracy has not yet been systematically tested. We performed Monte Carlo simulations where samples with pre-specified population correlations between the…

There exists a variety of interval estimators for the overall treatment effect in a random-effects meta-analysis. A recent literature review summarizing existing methods suggested that in most situations, the Hartung-Knapp/Sidik-Jonkman (HKSJ) method was preferable. However, a quantitative comparison of those methods in a common simulation study…

Conventional estimators for indirect effects using a difference in coefficients and product of coefficients produce the same results for continuous outcomes. However, for binary outcomes, the difference in coefficient estimator systematically underestimates the indirect effects because of a scaling problem. One solution is to standardize…

Given two 2-level factors of interest, a 2[superscript 2] split-plot design (a) takes each of the 2 [superscript 2] = 4 possible factorial combinations as a treatment, (b) identifies one factor as `whole-plot,' (c) divides the experimental units into blocks, and (d) assigns the treatments in such away that all units within the same block receive…

This article discusses the estimation of a population proportion, using the auxiliary information available, which is incorporated into the estimation procedure by a probit model fit. Three probit regression estimators are considered, using model-based and model-assisted approaches. The theoretical properties of the proposed estimators are derived…

Recent years have seen increasing interest in incorporating resampling methods into introductory statistics courses and the high school mathematics curriculum. While the use of permutation tests for data from experiments is a step forward, the use of simple bootstrap methods for sampling situations is more problematical. This article demonstrates…

The finite population correction (FPC) factor is often used to adjust variance estimators for survey data sampled from a finite population without replacement. As a replicated resampling approach, the jackknife approach is usually implemented without the FPC factor incorporated in its variance estimates. A paradigm is proposed to compare the…

This publication describes the methods and procedures used for the 2008/18 Baccalaureate and Beyond Longitudinal Study (B&B:08/18). The B&B graduates, who completed the requirements for a bachelor's degree during the 2007-08 academic year, were first surveyed as part of the 2008 National Postsecondary Student Aid Study (NPSAS:08), and then…

This report describes the design, methods, and results of the 2016/20 Baccalaureate and Beyond Longitudinal Study (B&B:16/20) conducted by the U.S. Department of Education's National Center for Education Statistics (NCES). It is the second follow-up with a cohort of bachelor's degree recipients originally identified during the 2015-16 National…

Identifying mediators in variable chains as part of a causal mediation analysis can shed light on issues of causation, assessment, and intervention. However, coefficients and effect sizes in a causal mediation analysis are nearly always small. This can lead those less familiar with the approach to reject the results of causal mediation analysis.…