This study presents several multivariate Generalizability theory designs for analyzing automatic item-generated (AIG) based test forms. The study used real data to illustrate the analysis procedure and discuss practical considerations. We collected the data from two groups of students, each group receiving a different form generated by AIG. A…

Reporting standard errors of equating has been advocated as a standard practice when conducting test equating. The two most widely applied procedures for standard errors of equating including the bootstrap method and the delta method are either computationally intensive or confined to the derivations of complicated formulas. In the current study,…

The use of large-scale assessments (LSAs) in education has grown in the past decade though analysis of LSAs using multilevel models (MLMs) using R has been limited. A reason for its limited use may be due to the complexity of incorporating both plausible values and weighted analyses in the multilevel analyses of LSA data. We provide additional…

It is common to find mixed-format data results from the use of both multiple-choice (MC) and constructed-response (CR) questions on assessments. Dealing with these mixed response types involves understanding what the assessment is measuring, and the use of suitable measurement models to estimate latent abilities. Past research in educational…

We provide a novel characterization of augmented balancing weights, also known as automatic debiased machine learning (AutoDML). These popular "doubly robust" or "de-biased machine learning estimators" combine outcome modeling with balancing weights -- weights that achieve covariate balance directly in lieu of estimating and…

The MacArthur-Bates Communicative Development Inventories (CDI) are widely used, parent-report instruments of language acquisition. Here, we focus on the word-inventory sections of the instruments, and show two different approaches to modeling CDI data, based on real-world needs. First, we show that Words & Gestures data collected…

We report findings of this replication study, examining the internalizing subscale (SRSS-I4) of the revised version of the Student Risk Screening Scale for Internalizing and Externalizing behavior (SRSS-IE 9) and the internalizing subscale of the Teacher Report Form (TRF). Using the sample from 13 elementary schools across three U.S. states with…

Cognitive ability is closely associated with the acquisition of programming skills, and enhancing learners' cognitive ability is a crucial factor in improving the efficacy of programming education. Adaptive feedback strategies can provide learners with personalized support based on their learning context, which helps to stimulate their interest…

Although educational research and evaluation generally occur in multilevel settings, many analyses ignore cluster effects. Neglecting the nature of data from educational settings, especially in non-randomized experiments, can result in biased estimates with long-term consequences. Our manuscript improves the availability and understanding of…

Generative AI (GenAI) has the potential to revolutionize the analysis of educational data, significantly impacting learning analytics (LA). This study explores the capability of non-experts, including administrators, instructors, and students, to effectively use GenAI for descriptive LA tasks without requiring specialized knowledge in data…

Students with and without disabilities may be educated across various service delivery models (SDMs): general education, cotaught, pull-out, and self-contained. Still, evidence for their relative effectiveness at scale remains limited. Using longitudinal administrative data from Indiana, we measured the effect of different SDMs on test scores,…

Machine Learning concept learns from experiences, inferences and conceives complex queries. Machine learning techniques can be used to develop the educational framework which understands the inputs from students, parents and with intelligence generates the result. The framework integrates the features of Machine Learning (ML), Explainable AI (XAI)…

Technical difficulties occasionally lead to missing item scores and hence to incomplete data on computerized tests. It is not straightforward to report scores to the examinees whose data are incomplete due to technical difficulties. Such reporting essentially involves imputation of missing scores. In this paper, a simulation study based on data…

Longitudinal data analysis has been widely employed to examine between-individual differences in within-individual changes. One challenge of such analyses is that the rate-of-change is only available indirectly when change patterns are nonlinear with respect to time. Latent change score models (LCSMs), which can be employed to investigate the…

This work proposes relevant ingredients to highlight the factors with significant impact on efficiency of research structures in higher education. The ingredients in question include methods and their implementation taking into account the choice and operationalization of factors as well as options of the retained methods. The methods employed in…