This report explores an approach to item development and psychometric modeling which explicitly incorporates knowledge about the mental models used by examinees in the solution of items into a psychometric model that characterize performances on a test, as well as incorporating that knowledge into the item development process. The paper focuses on…

The vocabulary used in textbooks and standardized tests for English as a second language (ESL) is examined to determine how much variation exists. Word lists were compared for six difficulty levels in five frequently-used ESL textbook series (a total of 30 books). Three series of standardized proficiency/placement tests at different levels were…

Multidimensional item difficulty (MID) is proposed as a means of describing test items which measure more than one ability. With mathematical story problems, for instance, both mathematical and verbal skills are required to obtain a correct answer. The proposed measure of MID is based upon three general assumptions: (1) the probability of…

A Monte Carlo study of five indices of dimensionality of binary items used a computer model that allowed sampling of both items and people. Five parameters were systematically varied in a factorial design: (1) number of common factors from one to five; (2) number of items, including 20, 30, 40, and 60; (3) sample sizes of 125 and 500; (4) nearly…

Specific questions addressed in this study include how time limits affect a test's construct and predictive validities, how time limits affect an examinee's time allocation and test performance, and whether the assumption about how examinees answer items is valid. Interactions involving an examinee's sex and age are studied. Two parallel forms of…

The work presented in this paper defined conceptually the concepts of multidimensional discrimination and information, derived mathematical expressions for the concepts for a particular multidimensional item response theory (IRT) model, and applied the concepts to actual test data. Multidimensional discrimination was defined as a function of the…

The effects of feedback about the ratings of other judges on subsequent ratings using the Nedelsky method and the ability of judges to retain or eliminate options in a manner consistent with the judgments of minimally competent examinees were studied using data from a basic algebra examination administered to 227 college students in 1987. The…

The South Pacific Commission's English reading curriculum, designed to accompany and reinforce the Tate English oral skills series, is critiqued in terms of curriculum design and implementation in schools in the Micronesian region. The construction and emphases of the classroom and teacher materials are described, and an analysis is given of the…

Instructional sensitivity indices are compared from four perspectives: (1) criterion-referenced testing; (2) classical test theory; (3) item response theory; and (4) Bayesian theory. Instructional sensitivity is defined as the tendency of a test item to vary in difficulty as a function of instruction. The instructional sensitivity of the items in…

Questionnaires were completed by 62 college professors in a study to determine the specific reading and listening comprehension difficulties students were having in various introductory courses (including humanities, natural and social sciences, and the technical professions), and to determine the instructional techniques used by professors to…

This study examined the difficulty level of essay topics used in the large-scale assessment of writing in relation to five different scoring models, and sought to determine what effects the scoring models would have on passing rates. In model one, examinee's score is the direct result of a score assigned by the reader or the sum of scores assigned…

The effect of increased writing or planning time on a test of basic college level writing ability was studied. The essay portion of the New Jersey College Basic Skills Placement Test was given to students in nine New Jersey public colleges and three New Jersey public high schools. Each student wrote two essays on two different topics. The first…

Three different item-bank maintenance procedures were compared for the purpose of investigating their cumulative effects on a series of longitudinal equatings of tests measuring the same characteristic. Item data used resulted from Rasch one-parameter latent trait test calibrations. Comparison of the results in the various final banks were made.…

A case study of the Florida Teacher Certification Examination (FTCE) program was described to assist others launching the development of large scale item banks. FTCE has four subtests: Mathematics, Reading, Writing, and Professional Education. Rasch calibrated item banks have been developed for all subtests except Writing. The methods used to…

The feasibility of creating an item bank from a teacher-made test was examined in two comparable sections of a graduate-level introductory measurement course. The 67-item midterm examination contained multiple-choice and master matching items, which required higher level cognitive processes such as application and analysis. The feasibility of…