Over the past decade, statisticians have developed methods to improve generalizations from nonrandom samples using propensity score methods. While these methods contribute to generalization research, their effectiveness is limited by small sample sizes. Small area estimation is a class of model-based methods that address the imprecision due to…

Purpose: Many studies using machine learning (ML) in speech, language, and hearing sciences rely upon cross-validations with single data splitting. This study's first purpose is to provide quantitative evidence that would incentivize researchers to instead use the more robust data splitting method of nested k-fold cross-validation. The second…

Researchers often have hypotheses concerning the state of affairs in the population from which they sampled their data to compare group means. The classical frequentist approach provides one way of carrying out hypothesis testing using ANOVA to state the null hypothesis that there is no difference in the means and proceed with multiple comparisons…

Combining observational and experimental data for causal inference can improve treatment effect estimation. However, many observational data sets cannot be released due to data privacy considerations, so one researcher may not have access to both experimental and observational data. Nonetheless, a small amount of risk of disclosing sensitive…

In mixed methods approaches, statistical models are used to identify "nested" cases for intensive, small-n investigation for a range of purposes, including notably the examination of causal mechanisms. This article shows that under a commonsense interpretation of causal effects, large-n models allow no reliable conclusions about effect…

Cross-validation is a common statistical procedure applied to problems that are otherwise computationally intractable. It is often employed to assess the effectiveness of prediction procedures. In this report, cross-validation is discussed in terms of "U"-statistics. This approach permits consideration of the statistical properties of…

Psychology research often focuses on interactions, and this has deep implications for inference from non-representative samples. For the goal of estimating average treatment effects, we propose to fit a model allowing treatment to interact with background variables and then average over the distribution of these variables in the population. This…

Kelley's Discrimination Index (DI) is a simple and robust, classical non-parametric short-cut to estimate the item discrimination power (IDP) in the practical educational settings. Unlike item-total correlation, DI can reach the ultimate values of +1 and -1, and it is stable against the outliers. Because of the computational easiness, DI is…

Periodically, articles reporting research on student ratings of instruction (SRI), aka student evaluations of teaching, appear in the higher-education press. This literature often summarizes studies that challenge the validity and reliability of SRI. However, before drawing a conclusion about a quantitative study touted in the media, readers…

In this article, first we obtained the correct mean square error expression of Gupta and Shabbir's linear weighted estimator of the ratio of two population proportions. Later we suggested the general class of ratio estimators of two population proportions. The usual ratio estimator, Wynn-type estimator, Singh, Singh, and Kaur difference-type…

Recent methods have sought to improve precision in randomized controlled trials (RCTs) by utilizing data from large observational datasets for covariate adjustment. For example, consider an RCT aimed at evaluating a new algebra curriculum, in which a few dozen schools are randomly assigned to treatment (new curriculum) or control (standard…

Teachers engaging in research has been discussed and carried out under the heuristics and methodologies of action research (Manfra, 2019; Pine, 2009). A typical action research project might involve an individual teacher studying the effectiveness of a specific instructional strategy like having students preview content before receiving direct…

Young children often endorse explanations of the natural world that appeal to functions or purpose--for example, that rocks are pointy so animals can scratch on them. By contrast, most Western-educated adults reject such explanations. What accounts for this change? We investigated 4- to 5-year-old children's ability to generalize the form of an…

While other fields such as statistics and education have examined various issues with quantitative work, few studies in physics education research (PER) have done so. We conducted a two-phase study to identify and to understand the extent of these issues in quantitative PER. During phase 1, we conducted a focus group of three experts in this area,…

The understanding and generalisation of causality are important thinking abilities, as they form the basis for a person's activity. Researchers exploring these abilities do not have a unified opinion regarding the age of children when they develop causative understanding and its determinant factors (e.g. age, prior knowledge, the content of a…