A procedure for estimating the reliability of test scores in the context of ecological momentary assessment (EMA) was proposed to take into account the characteristics of EMA measures. Two commonly used test scores in EMA were considered: the aggregated score (AGGS) and the within-person centered score (WPCS). Conceptually, AGGS and WPCS represent…

Reliabilities of the two most widely used intraindividual variability indicators, "ISD[superscript 2]" and "ISD", are derived analytically. Both are functions of the sizes of the first and second moments of true intraindividual variability, the size of the measurement error variance, and the number of assessments within a burst. For comparison,…

Factor score estimation is a controversial topic in psychometrics, and the estimation of factor scores from exploratory factor models has historically received a great deal of attention. However, both confirmatory factor models and the existence of missing data have generally been ignored in this debate. This article presents a simulation study…

Diagnostic scores are of increasing interest in educational testing due to their potential remedial and instructional benefit. Naturally, the number of educational tests that report diagnostic scores is on the rise, as are the number of research publications on such scores. This article provides a critical evaluation of diagnostic score reporting…

This study uses within-study comparisons to assess the relative importance of covariate choice, unreliability in the measurement of these covariates, and whether regression or various forms of propensity score analysis are used to analyze the outcome data. Two of the within-study comparisons are of the four-arm type, and many more are of the…

A method for point and interval estimation of change in criterion validity of multiple-component measuring instruments as a result of revision is outlined. The procedure is developed within the framework of covariance structure modeling, which complements earlier methods for testing change in composite reliability due to addition or deletion of…

Second-order latent growth curve models (S. C. Duncan & Duncan, 1996; McArdle, 1988) can be used to study group differences in change in latent constructs. We give exact formulas for the covariance matrix of the parameter estimates and an algebraic expression for the estimation of slope differences. Formulas for calculations of the required sample…

Outlines a correlation structures modeling approach to the study of stability in reliability of multiple, repeatedly administered measures and illustrates the method on data from a fluid intelligence study (P. Bates and others, 1986). The method is also applicable when examining relationships between model parameters across all variables.…

Proposes a test for change in composite reliability as a result of scale revision or modification. The test, which is useful for examining whether a revision changes the population reliability of a scale undergoing development, is illustrated with two examples. (SLD)

The population discrepancy between Cronbach's Coefficient Alpha (L. Cronbach, 1951) and scale reliability with fixed congeneric measure, uncorrelated errors, and sampling of subjects was studied. The difference is expressed in terms of the individual component violations of the assumption of equal tau-equivalence that is necessary and sufficient…

Sample size recommendations in confirmatory factor analysis (CFA) have recently shifted away from observations per variable or per parameter toward consideration of model quality. Extending research by Marsh, Hau, Balla, and Grayson (1998), simulations were conducted to determine the extent to which CFA model convergence and parameter estimation…

The procedures yielding confidence intervals for maximized alpha coefficients of Joe and Woodward are reviewed. Confidence interval procedures of Whalen and Masson are next reviewed. Results are then presented of a Monte Carlo investigation of the procedures. (Author/CTM)

The stability of dimensions extracted from a body of free response data was studied using 1,523 expressions of concern and questions raised by 271 elderly persons and analyzed by 2 groups of experimenters. The structures of resulting multidimensional configurations obtained by the 2 groups were identical. (SLD)

Results showed that the canonical correlations are very stable upon replication. The results also indicated that there is no solid evidence for concluding that components are superior to the coefficients, at least not in terms of being more reliable. (Author/BJG)

Proposes an analytic approach to standard error and confidence interval estimation of scale reliability with fixed congeneric measures. The method is based on a generally applicable estimator stability evaluation procedure, the delta method. The approach, which combines wide-spread point estimation of composite reliability in behavioral scale…