This paper presents an argument against the wider adoption of complex forms of data analysis, using multi-level modeling (MLM) as an extended case study. MLM was devised to overcome some deficiencies in existing datasets, such as the bias caused by clustering. The paper suggests that MLM has an unclear theoretical and empirical basis, has not led…

Univariate latent growth modeling (LGM) of composites of multiple items (e.g., item means or sums) has been frequently used to analyze the growth of latent constructs. This study evaluated whether LGM of composites yields unbiased parameter estimates, standard errors, chi-square statistics, and adequate fit indexes. Furthermore, LGM was compared…

A person fit test based on the Lagrange multiplier test is presented for three item response theory models for polytomous items: the generalized partial credit model, the sequential model, and the graded response model. The test can also be used in the framework of multidimensional ability parameters. It is shown that the Lagrange multiplier…

High quality assessment practice is expected to yield valid and useful score-based interpretations about what the examinees know and are able to do with respect to a defined target domain. Given this assertion, the article presents a framework based on the "unified view of validity," advanced by Cronbach and Messick over two decades ago, to assist…

We investigated language variation and score variation in the testing of English language learners, native Spanish speakers. We gave students the same set of National Assessment of Educational Progress mathematics items in both their first language and their second language. We examined the amount of score variation due to the main and interaction…

This report provides national data on the availability and use of educational technology in public elementary and secondary schools during fall 2008. The data are the results of a national school-level survey that is one of a set that includes district, school, and teacher surveys on educational technology. Every year between 1994 and 2005 (with…

This article illustrates the use of an explanatory item response modeling (EIRM) approach in the context of measuring group differences in science achievement. The distinction between item response models and EIRMs, recently elaborated by De Boeck and Wilson (2004), is presented within the statistical framework of generalized linear mixed models.…

The dependability of externalizing behavior composites and subscale scores from the Behavior Assessment System for Children, Second Edition, Teacher Rating Scale-Child (Reynolds & Kamphaus, 2004) and the Achenbach System of Empirically Based Assessment, Teacher's Report Form for Ages 6-18 (Achenbach & Rescorla, 2001) was investigated.…

The purpose of this study is to test the validity of scores on the Homework Management Scale (HMS) using 681 rural and 306 urban high school students. Based on a randomized split of the rural sample, the author first conducts exploratory factor analysis on Group 1 (n = 341) and confirmatory factor analysis on Group 2 (n = 340). The results reveal…

Policymakers have been puzzled to observe that food stamp households appear more likely to be food insecure than observationally similar eligible nonparticipating households. We reexamine this issue allowing for nonclassical reporting errors in food stamp participation and food insecurity. Extending the literature on partially identified…

Examined the standard error for short forms of Wechsler's scales with deviant subjects (N=2000). Demonstrated that the standard error of estimate of a short form for the standardization sample is an excellent approximation to the standard error of a predicted IQ for a new, even markedly deviant, subject. (LLL)

This paper suggests that although data from a homogenous sample might yield less reliable scores than did an inducted sample, these data should not be discarded until further examination of the data is conducted. The paper presents two statistics for monitoring data homogeneity and one statistic for correcting alpha when homogeneity is large. The…

This paper reviews the concept of experimentwise Type I error. While "testwise" alpha refers to the probability of making a Type I error for a single hypothesis test, "experimentwise" error refers to the probability of having made a Type I error anywhere within the study. Experimentwise error concerns are the basis for two…

Provides procedures for determining identified parameters, finding constraints on the covariances, and checking equivalence in acyclic (recursive) linear path models with correlated error terms (disturbances) by inspection of the path equations, aided by simple recursions. (SLD)

Derives the asymptotic standard errors and intercorrelations for several matrix correlations assuming multivariate normality for manifest variables and derives the asymptotic standard errors of the matrix correlations for two factor-loading matrices. (SLD)