Diagnostic scores are of increasing interest in educational testing due to their potential remedial and instructional benefit. Naturally, the number of educational tests that report diagnostic scores is on the rise, as are the number of research publications on such scores. This article provides a critical evaluation of diagnostic score reporting…

A linear combination of a set of measures is often sought as an overall score summarizing subject performance. The weights in this composite can be selected to maximize its reliability or to maximize its validity, and the optimal choice of weights is in general not the same for these two optimality criteria. We explore several relationships…

Evaluated external assessor ratings of the quality of the research team and the proposed research for proposals submitted to the Australian Research Council large grants program. Data from 313 evaluations show that the reliability of the assessments varied systematically with the number of assessors and the score considered. Discusses implication…

Presents a validation procedure for cognitive structures that is based on structural equation modeling of cognitive subordination relationships among test items. Illustrates the method using scores of 278 ninth graders on an algebra test and shows results for the same test when the linear logistic test model is used. (SLD)

Results of a reanalysis of previously published data (B. M. Byrne, 1989) support the correlated uniqueness model, diagnostic tests of the validity of confirmatory factor analysis (CFA), multitrait multimethod (MTMM) solutions, inclusion of external validity in MTMM design, and application of factorial invariance to test stability of CFA-MTMM…