Model fit indices are being increasingly recommended and used to select the number of factors in an exploratory factor analysis. Growing evidence suggests that the recommended cutoff values for common model fit indices are not appropriate for use in an exploratory factor analysis context. A particularly prominent problem in scale evaluation is the…

Sparse-rated data are common in operational performance-based language tests, as an inevitable result of assigning examinee responses to a fraction of available raters. The current study investigates the precision of two generalizability-theory methods (i.e., the rating method and the subdividing method) specifically designed to accommodate the…

Multiple regression is frequently used across the various social sciences to analyze cross-sectional data. However, it can often times be challenging to justify the assumption of common regression coefficients across all respondents. This manuscript presents a heterogeneous Bayesian regression model that enables the estimation of…

A number of psychometricians have argued for the use of parallel analysis to determine the number of factors. However, parallel analysis must be viewed at best as a heuristic approach rather than a mathematically rigorous one. The authors suggest a revision to parallel analysis that could improve its accuracy. A Monte Carlo study is conducted to…

The logistic person response function (PRF) models the probability of a correct response as a function of the item locations. Reise (2000) proposed to use the slope parameter of the logistic PRF as a person-fit measure. He reformulated the logistic PRF model as a multilevel logistic regression model and estimated the PRF parameters from this…

This study presents a new approach to estimation of a nonlinear growth curve component with fixed and random effects in multilevel modeling. This approach can be used to estimate change in longitudinal data, such as day-of-the-week fluctuation. The motivation of the new approach is to avoid spurious estimates in a random coefficient regression…

The purpose of this study was to investigate the application of the parallel analysis (PA) method for choosing the number of factors in component analysis for situations in which data are dichotomous or ordinal. Although polychoric correlations are sometimes used as input for component analyses, the random data matrices generated for use in PA…

Score limitation at the top of a scale is commonly termed "ceiling effect." Ceiling effects can lead to serious artifactual parameter estimates in most data analysis. This study examines the consequences of ceiling effects in longitudinal data analysis and investigates several methods of dealing with ceiling effects through Monte Carlo simulations…

One major challenge in using group projects to assess student learning is accounting for the differences of contribution among group members so that the mark assigned to each individual actually reflects their performance. This research addresses the validity of grading group projects by evaluating different methods that derive individualized…

This article introduces a bootstrap generalization to the Modified Parallel Analysis (MPA) method of test dimensionality assessment using factor analysis. This methodology, based on the use of Marginal Maximum Likelihood nonlinear factor analysis, provides for the calculation of a test statistic based on a parametric bootstrap using the MPA…

Recent work has shown that multidimensionally scoring responses from different tests can provide better ability estimates. For educational assessment data, applications of this approach have been limited to binary scores. Of the different variants, the de la Torre and Patz model is considered more general because implementing the scoring procedure…

Monte Carlo results were used to evaluate procedures for discriminating between groups. A multiple testing version of Hotelling's T-squared and the Bonferroni procedure were most powerful for detecting at least one true difference, depending on conditions examined. A multiple Bonferroni procedure was superior in power for detecting all true…

The problem of comparing proportions when some data are missing is investigated, and determination is made of what statistical techniques are appropriate under each of several probability models describing the observations likely to be missing. Monte Carlo methods were used to investigate the properties of standard estimators under each of the…

A new test assembly algorithm based on a Monte Carlo random search is presented in this article. A major advantage of the Monte Carlo test assembly over other approaches (integer programming or enumerative heuristics) is that it performs a uniform sampling from the item pool, which provides every feasible item combination (test) with an equal…

Within-class regression is a method, developed in this paper, of comparing a large number of nonequivalent groups. This study indicated that within-class regression was a less biased method of data analysis and will yield more accurate estimates of treatment effects than analysis of covariance. (PN)