This article describes some of the programming improvements made to a computer-based instructional support system after it was operating as had been envisioned during its design. Changes were made to accommodate differences between the original objectives and what users discovered they really wished to do. (Author/BD)

Instead of shying away from problem solving items, testing specialists should devise and offer guidelines for construction and scoring problem solving items. (Editor/LBH)

This document contains the framework and a set of recommendations for the 2005 NAEP mathematics assessment. It includes descriptions of the mathematical content of the test, the types of test questions, and recommendations for administration of the test. In broad terms, this framework attempts to answer the question: What mathematics should be…

This study represented a first attempt to evaluate the impact of local item dependence (LID) for Item Response Theory (IRT) scoring in computerized adaptive testing (CAT). The most basic CAT design and a simplified design for simulating CAT item pools with varying degrees of LID were applied. A data generation method that allows the LID among…

A study was conducted to identify local item dependence (LID) in the context-dependent item sets used in an examination prepared for use in an introductory university physics class and to assess the effects of LID on estimates of the reliability and standard error of measurement. Test scores were obtained for 487 students in the physics class. The…

This paper makes two criticisms of dichotomously scored instruments. One is that dichotomous scoring restrains the scores to ipsative measures that should not be compared, and the other is that dichotomous scoring ignores the strength with which a subject endorses a response so that the resulting count may imply a different construct measure from…

This document is a scoring guide that presents the Teacher-candidates' Overall Program Portfolio Scoring: Systematic, Comprehensive and Hierarchical Evaluative Measures of Excellence (TOPPS SCHEME) approach to scoring teacher candidates' program portfolios. The guide facilitates systematic, comprehensive, and hierarchical means of measuring and…

Three groups of persons are involved in the testing enterprise: test producers, test users, and test takers. A wide literature is available to guide the first two groups, but only recently have measurement professionals considered the interests of test takers in any careful way. The content of this chapter is presented as a set of 26…

The purpose of this study was to investigate the effect of a Rasch-based procedure to calibrate responses for funding applications. The data set included 112 proposals and 66 readers, who independently scored randomly assigned proposals using a scoring instrument. The data were analyzed using FACETS (Linacre, 1999). The analysis indicated that the…

Outside of large-scale testing programs, the computerized adaptive test (CAT) has thus far had only limited impact on measurement practice. In smaller-scale testing contexts, limited data are often available, which precludes the establishment of calibrated item pools for use by traditional (i.e., item response theory (IRT) based) CATs. This paper…

The conclusion of the present experiment was that prior response increases item consistency, but only when the prior experience involves the same item with the same instructions. (Authors)

The findings point to an urgent need for modification in the administration and scoring of the WISC. Institutions responsible for the training of psychologists need to provide extensive experience in all phases of scoring. Psychologists should be urged to recheck the scoring of each protocol they complete. In particular, all clerical work should…