The controversy regarding reverse or negatively-worded survey stems has been around for several decades. The practice has been used to guard against acquiescent or response set behaviors. A 20-item, 5-point Likert item survey was designed and the stems and response sets were varied in a 2 by 3 design. One independent variable was type of item…

This paper presents some Bayesian theories of simultaneous optimization of decision rules for test-based decisions. Simultaneous decision making arises when an institution has to make a series of selection, placement, or mastery decisions with respect to subjects from a population. An obvious example is the use of individualized instruction in…

This paper evaluated multidimensional linking procedures with which multidimensional test data from two separate calibrations were put on a common scale. Data were simulated with known ability distributions varying on two factors which made linking necessary: mean vector differences and variance-covariance (v-c) matrix differences. After the…

In the traditional Angoff Standard Setting Method, experts are instructed to predict the possibility that a randomly selected, hypothetical minimally competent candidate will be able to answer each multiple choice question in the test correctly. These item performance estimates are averaged across panelists and aggregated to determine the minimum…

Item response models can be applied in many test equating situations by making strong statistical assumptions. Thus, studying the robustness of the models to violations of the assumptions and investigating model-data fit are essential in all item response theory (IRT) equating applications (M. Kolen and R. Brennan, 1995). Previous studies dealing…

This paper provides a review of procedures for detection of differential item functioning (DIF) for item response theory (IRT) and observed score methods for the graded response model. In addition, data from a test anxiety scale were analyzed to examine the congruence among these procedures. Data from Nasser, Takahashi, and Benson (1997) were…

Test data generated according to two different multidimensional item response theory (IRT) models were compared at both the item response level and the test score level to determine whether measurable differences between the models could be detected when the data sets were constrained to be equivalent in terms of item "p"-values. The…

This paper compares the use of multiple pools versus a single pool with respect to test security against large-scale item sharing among some examinees in a computer-based test, under the assumption that a randomized item selection method is used. It characterizes the conditions under which employing multiple pools is better than using a single…

The validity studies planned for the Test Design activities deal primarily with the appropriateness of items generated for a domain. Previous exploratory work in the field related to overall test content appropriateness ratings has not been satisfactory. Studies which are solely based on correlational data suffer from confounding with…

Three formulae developed to correct item-total correlations for spuriousness were evaluated. Relationships among corrected, uncorrected, and item-remainder correlations were determined by computing sets of mean, minimum, and maximum deviation coefficients and Spearman rank correlations for nine test lengths. (Author/JKS)

Two exam questions are presented. One suitable for advanced undergraduate or beginning graduate courses in organic chemistry, is on equivalent expressions for the description of several pericyclic reactions. The second, for general chemistry students, asks for an estimation of the rate of decay of a million-year-old Uranium-238 sample. (BB)

A modification of Hoyt's analysis of variance model for test analysis was proposed by Lu. A difficulty that may be encountered in using Lu's modification is examined, and a solution is proposed. (JKS)

Generalizations of the G index as a measure of similarity between persons beyond the dichotomous situation are discussed. An attempt is made to present a generalization that does not require dichotomization of the items for cases where the number of response alternatives may differ. (Author/JKS)

The standardization method for assessing unexpected differential item performance or differential item functioning is introduced. Findings of five studies are summarized, in which the statistical method of standardization is used to look for unexpected differences in item performance across different subpopulations of the Scholastic Aptitude Test.…

This article provides unbiased estimates of the proportion of items in an item domain that an examinee would answer correctly if every item were attempted, when a closed sequential testing procedure is used. (Author)