Given its centrality to assessment, until the concept of validity includes concern for racial justice, such matters will be seen as residing outside the "real" work of validation, rendering them powerless to count against the apparent scientific merit of the test. As the definition of validity has evolved, however, it holds great…

It has often been asserted that the reliability of a measure places an upper limit on its validity. This article demonstrates in theory that validity can rise when reliability declines, even when validity evidence is a correlation with an acceptable criterion. Whether empirical examples can actually be found is an open question. (SLD)

An analysis of data from the 1990 National Assessment of Educational Progress Trial State Assessment suggests that person-fit statistics may not provide additional information about results of psychometrically strong achievement tests. More research is needed before person-fit statistics can be used routinely in analysis of item response data.…

Statistical and theoretical issues that arise from assessing person-fit on measures of typical performance are discussed, including the frequent attenuation of detection of person-misfit, the need for methods of identifying sources of response aberrancy, and person-fit measures as moderators of trait-criterion relations. (SLD)

This overview of the role of error and tolerance for error in measurement asserts that the generic precision associated with a measurement procedure is defined as the root mean square error, or standard error, in some relevant population. This view of precision is explored in several applications of measurement. (SLD)

Application of person-fit statistics to cognitive diagnosis requires special efforts to detect normal and usual response patterns resulting from sources of misconception that are frequently observed among students. This study shows a solution for the problem by introducing an extension of a person-fit statistic developed by K. Tatsuoka (1985).…

When item response data do not satisfy the unidimensionality assumption, multidimensional item response theory (MIRT) should be used to model the item-examinee interaction. This article presents and discusses MIRT analyses designed to give better insight into what individual items are measuring. (SLD)

The following ways of linking results from distinct assessments to use them for multiple purposes are reviewed: (1) equating; (2) calibration; (3) statistical moderation; (4) prediction; and (5) social moderation. The characteristics of these methods, their requirements, and the comparative inferences they support are described. (SLD)

Applying the principles of latent trait analysis makes it possible to rank order profiles of students seeking college admission in terms of the adequacy of a single score. An illustration using 577 profiles shows that it is possible that only a subset of profiles may require qualitative analysis. (SLD)

Issues that must be addressed in the large-scale application of computerized adaptive testing are explored, including considerations of test design, scoring, test administration, item and item bank development, and other aspects of test construction. Possible solutions and areas in which additional work is needed are identified. (SLD)

Portfolios are the centerpiece of Vermont's statewide assessment program in mathematics. Portfolio scores in the first two years were not reliable enough to permit the reporting of student-level results, but increasing the number of readers or the number of portfolio pieces is not operationally feasible. (SLD)

It is proposed that criteria for evaluating the quality of performance should be defined, at least tentatively, during the initial design of a performance assessment. Six characteristics of sound criteria are (1) significance; (2) fidelity; (3) generalizability; (4) developmental appropriateness; (5) accessibility; and (6) utility. (SLD)

The nature of admissions tests and their intended uses are reviewed. Evidence regarding the validity of tests, their contributions to admissions decisions, the effects of coaching, and possible bias in the predictive meaning of test scores for women and minorities are discussed. (Author/SLD)

Four main approaches to customized testing are described, and their resulting scores' valid uses and interpretations are discussed. Customized testing can yield valid normative and curriculum-specific information, although cautious application is needed to avoid misleading inferences about student achievement. (SLD)

Issues pertaining to the quality of performance assessments, including reliability and validity, are discussed. The relatively limited generalizability of performance across tasks is indicative of the care needed to evaluate performance assessments. Quality control is an empirical matter when measurement is intended to inform public policy. (SLD)