We investigated the potential impact of differential item functioning (DIF) on group-level mean and standard deviation estimates using empirical and simulated data in the context of large-scale assessment. For the empirical investigation, PISA 2018 cognitive domains (Reading, Mathematics, and Science) data were analyzed using Jackknife sampling to…

Large-scale assessment survey (LSAS) data are collected via complex sampling designs with special features (e.g., clustering and unequal probability of selection). Multilevel models have been utilized to account for clustering effects whereas the probability weighting approach (PWA) has been used to deal with design informativeness derived from…

Probabilistic machine learning increasingly informs critical decisions in medicine, economics, politics, and beyond. To aid the development of trust in these decisions, we develop a taxonomy delineating where trust in an analysis can break down: (1) in the translation of real-world goals to goals on a particular set of training data, (2) in the…

Access to authentic research is limited at the 7-12 science education level. At the same time, many local restoration projects would benefit from, but don't have access to a long-term system of monitoring. This project seeks to unite those two needs by developing a protocol for 7-12 classrooms to be able to participate in authentic research…

This paper extends existing work on teacher weighting in student-centered surveys by looking into aspects of practical implementation of deriving and using weights for teacher-centered analysis in the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS). The formal…

Given two 2-level factors of interest, a 2[superscript 2] split-plot design (a) takes each of the 2 [superscript 2] = 4 possible factorial combinations as a treatment, (b) identifies one factor as `whole-plot,' (c) divides the experimental units into blocks, and (d) assigns the treatments in such away that all units within the same block receive…

Conventional optimal design frameworks consider a narrow range of sampling cost structures that thereby constrict their capacity to identify the most powerful and efficient designs. We relax several constraints of previous optimal design frameworks by allowing for variable sampling costs in cluster-randomized trials. The proposed framework…

This study addresses measurement issues around a standards-based content analysis of mathematics textbooks' coverage of standards for use in large-scale monitoring of standards implementation as proposed in a 2013 report by the National Research Council. An earlier study produced an exhaustive content analysis of textbooks using the 2012 Common…

Variability is common in language sample analysis (LSA), arising from personal factors such as age or level of education, or from factors within the text such as its length and purpose. Variability can affect interpretation of results in clinical practice and research studies, as well as the ability to detect change in individuals over time. This…

Computer simulation can be used to demonstrate why the unbiased sample variance uses degrees of freedom (n-1). This is first demonstrated for sampling from a normal random variable, and in additional simulations for some selected non-normal random variables, namely, chi-square and binomial.

In social and behavioral sciences, data are typically not normally distributed, which can invalidate hypothesis testing and lead to unreliable results when being analyzed by methods developed for normal data. The existing methods of generating multivariate non-normal data typically create data according to specific univariate marginal measures…

Online networks, as digital images reflecting every aspects of real world especially under the current context of information technology, have become the most massive and opulent data source these days, where vast amount of information keeps being produced and transmitted. Their rapid expansion in scale and data volume draws wide attention and…

With many pretreatment covariates and treatment factors, the classical factorial experiment often fails to balance covariates across multiple factorial effects simultaneously. Therefore, it is intuitive to restrict the randomization of the treatment factors to satisfy certain covariate balance criteria, possibly conforming to the tiers of…

As the integration of digital technologies continues to shape academic landscapes, assessing digital literacy in the context of academic writing becomes paramount. Several instruments and frameworks are available for measuring digital literacy and examining it from different perspectives; however, none are suitable for measuring the digital…

In recent years, harmonic regression models have been applied to implement quality control for educational assessment data consisting of multiple administrations and displaying seasonality. As with other types of regression models, it is imperative that model adequacy checking and model fit be appropriately conducted. However, there has been no…