Randomized A/B tests within online learning platforms enable us to draw unbiased causal estimators. However, precise estimates of treatment effects can be challenging due to minimal participation, resulting in underpowered A/B tests. Recent advancements indicate that leveraging auxiliary information from detailed logs and employing design-based…

A brief, practical overview of "design effects" (DEFFs) is presented for users of the results of sample surveys. The overview is intended to help such users to determine how and when to use DEFFs and to compute them correctly. DEFFs are needed only for inferential statistics, not for descriptive statistics. When the selections for…

This paper discusses the impact of sampling error on the construction of confidence intervals around effect sizes. Sampling error affects the location and precision of confidence intervals. Meta-analytic resampling demonstrates that confidence intervals can haphazardly bounce around the true population parameter. Special software with graphical…

In comparing measurement theories, it is evident that the awareness of the concept of measurement error during the time of Galileo has lead to the formulation of observed scores comprising a true score and error (classical theory), universe score and various random error components (generalizability theory), or individual latent ability and error…

This LISREL study examines the robustness of the maximum likelihood estimates under varying degrees of measurement model misspecification. A true model containing five latent variables (two endogenous and three exogenous) and two indicator variables per latent variable was used. Measurement model misspecification considered included errors of…

The serious problems associated with the use of stepwise methods are well documented. Various authors have leveled scathing criticisms against the use of stepwise techniques, yet it is not uncommon to find these methods continually employed in educational and psychological research. The three main problems with stepwise techniques are: (1)…

Though common structural equation modeling (SEM) methods are predicated upon the assumption of multivariate normality, applied researchers often find themselves with data clearly violating this assumption and without sufficient sample size to use distribution-free estimation methods. Fortunately, promising alternatives are being integrated into…

The purpose of this paper is to evaluate the use of MC2G program to teach certain topics in statistics education. MC2G is a program written in Pascal Delphi by Gordon Brooks of Ohio University based on Monte Carlo studies. MC2G provides students opportunity to practice important topics in an introductory statistics course, such as power, Type I…

A component analytic method for analyzing multivariate longitudinal data is presented that does not make strong assumptions about the structure of the data. Central to the method are the facts that components are derived as linear composites of the observed or manifest variables and that the components must provide an adequate representation of…

With the help of widely available microcomputers, it is possible to demonstrate certain statistical phenomena which students of statistics are usually expected to take on faith. Two demonstrations are described. In the first demonstration, three common types of sampling (simple random, biased, and stratified-random) are used to compare statistics…

Ways in which the Statistical Package for the Social Sciences (SPSS) can be used to perform some Rasch analyses are described in detail. It is shown how SPSS and a set of item calibrations can be used to estimate person abilities, standard errors of measurement, test characteristic curve, test information curve, classification consistency on a…

Some specific computer-based tools for collecting and examining audit trails, i.e., data that describe a learner's path through a computer-based instruction (CBI) lesson, are detailed. Data analysis and measurement issues related to audit trails are also discussed. A particular set of embedded computer-based tools currently being used at San Diego…

LISREL-type structural equation modeling is a powerful statistical technique that seems appropriate for social science variables which are complex and difficult to measure. The literature on the specification, estimation, and testing of such models is voluminous. The greatest proportion of this literature, however, focuses on the technical aspects…