This study presents several multivariate Generalizability theory designs for analyzing automatic item-generated (AIG) based test forms. The study used real data to illustrate the analysis procedure and discuss practical considerations. We collected the data from two groups of students, each group receiving a different form generated by AIG. A…

Purpose: A considerable number of screening and diagnostic tools for autism exist, but variability in these measures presents challenges to data harmonization and the comparability and generalizability of findings. At the same time, there is a movement away from autism symptomatology to stances that capture heterogeneity and appreciate diversity.…

Value added models (VAMs) attempt to estimate the causal effects of teachers and schools on student test scores. We apply Generalizability Theory to show how estimated VA effects depend upon the selection of test items. Standard VAMs estimate causal effects on the items that are included on the test. Generalizability demands consideration of how…

This article explores the potential of artificial intelligence (AI) for academic advising. Specifically, it examines how AI-powered machine interpretation and data analysis can be used to deliver advising services anytime, anywhere. This system would eliminate the need for students to physically meet with advisors and could answer their…

Educational reform priorities such as emphasis on quantitative modelling (QM) have positioned undergraduate biology instructors as designers of QM experiences to engage students in authentic science practices that support the development of data-driven and evidence-based reasoning. Yet, little is known about how biology instructors adapt to the…

Researchers across many fields have called for greater attention to heterogeneity of treatment effects--shifting focus from the average effect to variation in effects between different treatments, studies, or subgroups. True heterogeneity is important, but many reports of heterogeneity have proved to be false, non-replicable, or exaggerated. In…

Purpose: Estimating the sequential associations between educators' and children's talk during language learning interactions requires careful consideration of factors that may impact measurement stability and resultant inferences. This research note will describe a preliminary study that used generalizability theory to understand the contribution…