Model fit indices are being increasingly recommended and used to select the number of factors in an exploratory factor analysis. Growing evidence suggests that the recommended cutoff values for common model fit indices are not appropriate for use in an exploratory factor analysis context. A particularly prominent problem in scale evaluation is the…

Knowledge tracing is a fundamental area of educational data modeling that aims at gaining a better understanding of the learning occurring in tutoring systems. Knowledge tracing models fit various parameters on observed student performance and are evaluated through several goodness of fit metrics. Fitted parameter values are of crucial interest in…

A common concern about the Beck Depression Inventory, Second edition (BDI-II) among researchers in the area of depression has long been the single-factor scoring scheme. Methods exist for making cross-sample comparisons of latent structure but tend to rely on estimation methods that can be imprecise and unnecessarily complex. This study presents a…

The method of least-squares (LS) has a built-in procedure for estimating the standard errors (SEs) of the adjustable parameters in the fit model: They are the square roots of the diagonal elements of the covariance matrix. This means that one can use least-squares to obtain numerical values of propagated errors by defining the target quantities as…

Conventional differential item functioning (DIF) detection methods (e.g., the Mantel-Haenszel test) can be used to detect DIF only across observed groups, such as gender or ethnicity. However, research has found that DIF is not typically fully explained by an observed variable. True sources of DIF may include unobserved, latent variables, such as…

Researchers use many different metrics for evaluation of performance of student models. The aim of this paper is to provide an overview of commonly used metrics, to discuss properties, advantages, and disadvantages of different metrics, to summarize current practice in educational data mining, and to provide guidance for evaluation of student…

This study investigated the performance of fit indexes in selecting a covariance structure for longitudinal data. Data were simulated to follow a compound symmetry, first-order autoregressive, first-order moving average, or random-coefficients covariance structure. We examined the ability of the likelihood ratio test (LRT), root mean square error…

Many methods exist for imputing missing data but fewer methods have been proposed to test the missing data mechanism. Little (1988) introduced a multivariate chi-square test for the missing completely at random data mechanism (MCAR) that compares observed means for each pattern with expectation-maximization (EM) estimated means. As an alternative,…

Statistical modeling of school effectiveness data was originally motivated by the dissatisfaction with the analysis of (school-leaving) examination results that took no account of the background of the students or regarded each school as an isolated unit of analysis. The application of multilevel analysis was generally regarded as a breakthrough,…

The distance between nonparametric and parametric item characteristic curves has been proposed as an index of goodness of fit in item response theory in the form of a root integrated squared error index. This article proposes to use the posterior distribution of the latent trait as the nonparametric model and compares the performance of an index…

Two-level data sets are frequently encountered in social and behavioral science research. They arise when observations are drawn from a known hierarchical structure, such as when individuals are randomly drawn from groups that are randomly drawn from a target population. Although 2-level data analysis in the context of structural equation modeling…

This study examined the efficacy of 4 different parceling methods for modeling categorical data with 2, 3, and 4 categories and with normal, moderately nonnormal, and severely nonnormal distributions. The parceling methods investigated were isolated parceling in which items were parceled with other items sharing the same source of variance, and…

A Rasch measurement model can be constructed to meet the requirements of rank ordered data. If multiple rankings of the same objects are available, then the parameters of the objects can be estimated, along with their standard errors and also with statistics summarizing the fit of the data to the measurement model. This paper summarizes the…

The Werts-Linn procedure for dealing with categorical errors of measurement in "Comments on Boyle's 'Path Analysis and Ordinal Data'" in The American Journal of Sociology, volume 76, number 6, May 1971, is shown to be inappropriate to the problem of ordered categories. (For related document, see TM 002 301.) (DB)

The logic of using a gain score approach versus longitudinal causal models is studied in this secondary analysis of a complex data base. The gain score model used by the Federal Reserve Bank and the School District of Philadelphia in their "What Works in Reading?" study is successively refined using the LISREL structural equation…