The Schwarz or Bayesian information criterion (BIC) is one of the most widely used tools for model comparison in social science research. The BIC, however, is not suitable for evaluating models with order constraints on the parameters of interest. This article explores two extensions of the BIC for evaluating order-constrained models, one where a…

BASIE is a framework for interpreting impact estimates from evaluations. It is an alternative to null hypothesis significance testing. This guide walks researchers through the key steps of applying BASIE, including selecting prior evidence, reporting impact estimates, interpreting impact estimates, and conducting sensitivity analyses. The guide…

The characteristic curve methods have been applied to estimate the equating coefficients in test equating under the graded response model (GRM). However, the approaches for obtaining the standard errors for the estimates of these coefficients have not been developed and examined. In this study, the delta method was applied to derive the…

The original and revised versions of the Internalized Homophobia Scale (IHP) were examined for gender invariance. The revised version was found to have a better fit, and passed the test of configural, metric, and scalar invariance. The revised version is recommended as a brief measure of internalized heterosexism.

Synthesizing findings about the indirect (mediated) effect plays an important role in determining the mechanism through which variables affect one another. This simulation study compared six methods for synthesizing indirect effects: correlation-based MASEM, parameter-based MASEM, marginal likelihood synthesis, an adjustment to marginal likelihood…

In recent years, improving the quantitative methods used to assess the effect of college, and particular college experiences, on student outcomes has received increased attention (e.g., Mayhew et al., 2016). In "How College Affects Students," Mayhew et al. (2016) highlighted the importance of issues of practical vs. statistical…

In this digital ITEMS module, Dr. Sue Lottridge, Amy Burkhardt, and Dr. Michelle Boyer provide an overview of automated scoring. Automated scoring is the use of computer algorithms to score unconstrained open-ended test items by mimicking human scoring. The use of automated scoring is increasing in educational assessment programs because it allows…

When the relationship between two physical variables, such as voltage and current, can be expressed as y = bx where b is a constant. b may be estimated by least squares, or by averaging the values of b obtained for each x-y data pair. We show for data gathered in an experiment, as well as through Monte Carlo simulation and mathematical analysis,…

The Bayesian way of accounting for the effects of error in the ability and item parameters in adaptive testing is through the joint posterior distribution of all parameters. An optimized Markov chain Monte Carlo algorithm for adaptive testing is presented, which samples this distribution in real time to score the examinee's ability and optimally…

The performance of statistical methods is often evaluated by means of simulation studies. In case of network meta-analysis of binary data, however, simulations are not currently available for many practically relevant settings. We perform a simulation study for sparse networks of trials under between-trial heterogeneity and including multi-arm…

Structural equation modeling (SEM) applications routinely employ a trilogy of significance tests that includes the likelihood ratio test, Wald test, and score test or modification index. Researchers use these tests to assess global model fit, evaluate whether individual estimates differ from zero, and identify potential sources of local misfit,…

This meta-analysis explores the correlation between self-assessment (SA) and language performance. Sixty-seven studies with 97 independent samples involving more than 68,500 participants were included in our analysis. It was found that the overall correlation between SA and language performance was 0.466 (p < 0.01). Moderator analysis was…

Model fit indices are being increasingly recommended and used to select the number of factors in an exploratory factor analysis. Growing evidence suggests that the recommended cutoff values for common model fit indices are not appropriate for use in an exploratory factor analysis context. A particularly prominent problem in scale evaluation is the…

Despite ongoing global efforts to combat discrimination, hate speech is a growing problem in many countries. Hate speech is also widespread among young people and thus also affects schools. Because of the negative consequences of hate speech for the targeted individuals and for society in general, schools and teachers are generally encouraged to…

This dissertation develops and applies sophisticated Item Response Theory (IRT) methods to address fundamental measurement challenges in cognitive testing, focusing on the Armed Services Vocational Aptitude Battery (ASVAB) data from the National Longitudinal Survey of Youth (NLSY). The first chapter implements a confirmatory multidimensional IRT…