Diagnostic scores are of increasing interest in educational testing due to their potential remedial and instructional benefit. Naturally, the number of educational tests that report diagnostic scores is on the rise, as are the number of research publications on such scores. This article provides a critical evaluation of diagnostic score reporting…

In DIF studies, a Type I error refers to the mistake of identifying non-DIF items as DIF items, and a Type I error rate refers to the proportion of Type I errors in a simulation study. The possibility of making a Type I error in DIF studies is always present and high possibility of making such an error can weaken the validity of the assessment.…

Despite high-stakes applications of assessment findings, assessment data are frequently collected in situations that are of low-stakes to examinees. Because low-stakes tests are of little consequence to the examinees, test-taking motivation and thus the validity of inferences drawn from unmotivated examinees' scores are of concern. The current…

Educational achievement is a very important issue for parents, teachers, and the government. An accurate measurement plays a very important role in evaluating achievement fairly, and, therefore, analysis methods have been developed considerably in recent years. Education based on long-time learning processes forms a fruitful base for item tests,…

Since the passage of the No Child Left Behind Act of 2001 (2002), there has been increased interest in using student achievement data (through standardized tests) to evaluate teacher effectiveness. Two U.S. Department of Education secretaries, Secretary Spellings and Secretary Duncan, have expressed interest in growth models and the need to…

In this commentary, the authors discuss some of the issues regarding the use of diagnostic classification models that practitioners should keep in mind. In the authors experience, these issues are not as well known as they should be. The authors then provide recommendations on diagnostic scoring.

This article describes the history, evidence warrants, and evolution of the Center for Research on Evaluation, Standards, and Student Testing's (CRESST) model-based assessments. It considers alternative interpretations of scientific or practical models and illustrates how model-based assessment addresses both definitions. The components of the…

A project was conducted to develop a model for evaluating placement testing in the North Carolina System of Community Colleges. Researchers at Central Piedmont Community College conducted a longitudinal study of students' progress through the college curriculum as it related to placement test scores. The following numbers of students comprised the…

Article discussed the disadvantages of student testing as a means of evaluating student progress in the classroom and suggested the use of a new model of assessment. Three steps intended for classroom diagnosis of students were described. (RK)

Background: In this article we address some of the challenges posed by the development of national assessment systems and discuss the need for high quality information on trends in attainment; support for school improvement processes and ways in which learning should be enhanced through valid assessment. Purpose: Key elements are explored,…

A structure for describing different approaches to testing is generated by identifying five dimensions along which tests differ: test uses, item generation, item revision, assessment of precision, and validation. These dimensions are used to profile tests of reading comprehension. Only norm-referenced achievement tests had an inference system…

Attempts to select empirically the optimal cutting score or criterion level for a test based on response data from validation samples of instructed and uninstructed students. This score maximizes the probability of correct mastery-nonmastery decisions (or minimizes the probability of incorrect decisions). (Author/RK)

Reviews research literature on comparability issues in computerized adaptive testing (CAT) and synthesizes issues specific to comparability and test security. Develops a framework for evaluating comparability that contains three categories of criteria: (1) validity; (2) psychometric property/reliability; and (3) statistical assumption/test…

This paper synthesizes ideas from the fields of graphical modeling and education testing, particularly item response theory (IRT) applied to computerized adaptive testing (CAT). Graphical modeling can offer IRT a language for describing multifaceted skills and knowledge, and disentangling evidence from complex performances. IRT-CAT can offer…

The stimulus for this colloquium was the convergence of several significant developments bearing on the construct validation of standardized tests and other assessment methods. Of these developments, some were fundamental to psychology as a science; others reflected socio-political pressures on measurement in education and employment. The ten…