One commonly used compromise standard-setting method is the Beuk (1984) method. A key assumption of the Beuk method is that the emphasis given to the pass rate and the percent correct ratings should be proportional to the extent that the panelists agree on their ratings. However, whether the slope of Beuk line reflects the emphasis that panelists…

Propensity score matching and weighting methods are applied to balance covariates and reduce selection bias in the analysis of observational study data, and ultimately estimate a treatment effect. We wish to evaluate the impact of a Supplemental Instruction (SI) program on student success in an Introductory Statistics course. In such student…

There is increasing interest in meta-analysis in different fields due to the need to combine the results of primary research. One of the crucial concepts in combining results is weighting. This study examines how Hunter and Schmidt's method, weighting by sample size; Hedges and Vevea's method, weighting by inverse variance; and Osburn and…

The EdSurvey package helps users download, explore variables in, extract data from, and run analyses on large-scale assessment data. The analysis functions in EdSurvey account for the use of plausible values for test scores, survey sampling weights, and their associated variance estimator. We describe the capabilities of the package in the context…

Modeling within-person change over time and between-person differences in change over time is a primary goal in prevention science. When modeling change in an observed score over time with multilevel or structural equation modeling approaches, each observed score counts toward the estimation of model parameters equally. However, observed scores…

Structural equation modeling (SEM) and path analysis using composite-scores are distinct classes of methods for modeling the relationship of theoretical constructs. The two classes of methods are integrated in the partial-least-squares approach to structural equation modeling (PLS-SEM), which systematically generates weighted composites and uses…

The article highlights the growing significance of flexible assessment in higher education as institutions adapt to the increasingly diverse needs of their student populations. The demand for customizable educational experiences, heightened by the COVID-19 pandemic, has made flexibility in assessment essential for sustaining and improving student…

Reducing free-riding is an important challenge for educators who use group projects. In this study, we measure students' preferences for group project characteristics and investigate if characteristics that better help to reduce free-riding become more important for students when stakes increase. We used a discrete choice experiment based on 12…

Equating the scores from different forms of a test requires collecting data that link the forms. Problems arise when the test forms to be linked are given to groups that are not equivalent and the forms share no common items by which to measure or adjust for this group nonequivalence. We compared three approaches to adjusting for group…

Structural equation modeling (SEM) has been deemed as a proper method when variables contain measurement errors. In contrast, path analysis with composite-scores is preferred for prediction and diagnosis of individuals. While path analysis with composite-scores has been criticized for yielding biased parameter estimates, recent literature pointed…

Distance learning has been adopted as an alternative learning strategy to the face-to-face teaching methodology. It has been largely implemented by many governments worldwide due to the spread of the COVID-19 pandemic and the implication in enforcing lockdown and social distancing. In emergency situations distance learning is referred to as…

Research studies show that students' grades are more predictive than test scores of their future academic success, including high school and post-secondary outcomes. Chicago Public Schools (CPS) recognizes the importance of grades through its longstanding "Bs or Better" campaign. Yet within CPS, boys' grades are consistently lower than…

We make a distinction between the operational practice of using an observed score to assess differential item functioning (DIF) and the concept of departure from measurement invariance (DMI) that conditions on a latent variable. DMI and DIF indices of effect sizes, based on the Mantel-Haenszel test of common odds ratio, converge under restricted…

Large-scale assessment survey (LSAS) data are collected via complex sampling designs with special features (e.g., clustering and unequal probability of selection). Multilevel models have been utilized to account for clustering effects whereas the probability weighting approach (PWA) has been used to deal with design informativeness derived from…

This dissertation consists of three chapters covering applications within Matching Theory. In the first chapter, we study the decentralized college admissions in the Kingdom of Saudi Arabia (KSA), where students submit their applications to each college separately, and colleges use a ranking based on weighted centralized test scores. Unlike the…