Large language models (LLMs) provide cost-effective but possibly inaccurate predictions of human behavior. Despite growing evidence that predicted and observed behavior are often not "interchangeable," there is limited guidance on using LLMs to obtain valid estimates of causal effects and other parameters. We argue that LLM predictions…

This paper aims to showcase the value of implementing a Bayesian framework to analyze and report results from international large-scale assessments and provide guidance to users who want to analyse ILSA data using this approach. The motivation for this paper stems from the recognition that Bayesian statistical inference is fast becoming a popular…

For randomized trials that use text as an outcome, traditional approaches for assessing treatment impact require that each document first be manually coded for constructs of interest by trained human raters. This process, the current standard, is both time-consuming and limiting: even the largest human coding efforts are typically constrained to…

Recent research suggests the benefits of supporting a progressive understanding of concepts of inference prior to the teaching of procedures and formal calculations through the study of informal statistical inference (ISI). To contribute to the growing knowledge about the early learning and teaching of statistics, particularly regarding the…

Structural equation modeling (SEM) is a statistical method widely used in educational research to investigate relationships between variables. SEM models are typically constructed based on theoretical foundations and assessed through fit indices. However, a well-fitting SEM model alone is not sufficient to verify the causal inferences underlying…

Of critical importance to education policy is monitoring trends in education outcomes over time. In the United States, the National Assessment of Educational Progress (NAEP) has provided long-term trend data since 1970; at the state/jurisdiction level, NAEP has provided long-term trend data since 1996. In addition to the national NAEP, all 50…

The synthetic control method (SCM) is a popular approach for estimating the impact of a treatment on a single unit in panel data settings. The "synthetic control" is a weighted average of control units that balances the treated unit's pre-treatment outcomes and other covariates as closely as possible. A critical feature of the original…

Randomized A/B tests within online learning platforms enable us to draw unbiased causal estimators. However, precise estimates of treatment effects can be challenging due to minimal participation, resulting in underpowered A/B tests. Recent advancements indicate that leveraging auxiliary information from detailed logs and employing design-based…

This report aims to showcase the value of implementing a Bayesian framework to analyse and report results from international large-scale surveys and provide guidance to users who want to analyse the data using this approach. The motivation for this report stems from the recognition that Bayesian statistical inference is fast becoming a popular…

Children have limited exposure to statistical concepts and processes, yet researchers have highlighted multiple benefits of experiences in which they design and/or engage informally with statistical modelling. A study was conducted with a classroom in which students developed and utilised data-based models to respond to the inquiry question,…

Multilevel modeling (MLM) is a statistical technique for analyzing clustered data. Despite its long history, the technique and accompanying computer programs are rapidly evolving. Given the complexity of multilevel models, it is crucial for researchers to provide complete and transparent descriptions of the data, statistical analyses, and results.…

Using simulation-based inference (SBI), such as randomization tests, as the primary vehicle for introducing students to the logic and scope of statistical inference has been advocated with the potential of improving student understanding of statistical inference and the statistical investigative process. Moving beyond the individual class…

Statistical analysis of categorical data often relies on multiway contingency tables; yet, as the number of categories and/or variables increases, the number of table cells with few (or zero) observations also increases. Unfortunately, sparse contingency tables invalidate the use of standard good-ness-of-fit statistics. Limited-information fit…

In today's age of information, the use of data is very powerful in making informed decisions. Data analytics is a field that is interested in identifying and interpreting trends and patterns within big data to make data-driven decisions. We focus on informal statistical inference and data modeling as a means of developing students' data analytics…

Bayesian item response theory (IRT) modeling stages include (a) specifying the IRT likelihood model, (b) specifying the parameter prior distributions, (c) obtaining the posterior distribution, and (d) making appropriate inferences. The latter stage, and the focus of this research, includes model criticism. Choice of priors with the posterior…