Educational item banking presents observers with a considerable paradox. The development of test items from scratch is viewed as wasteful, a luxury in times of declining resources. On the other hand, item banking has failed to become a mature technology despite large amounts of money and the efforts of talented professionals. The question of which…

Item parameter estimates vary for a variety of reasons, including estimation error, characteristics of the examinee samples, and context effects (e.g., item location effects, section location effects, etc.). Although we expect variation based on theory, there is reason to believe that observed variation in item parameter estimates exceeds what…

This paper extends the theory of conditional covariances to polytomous items. It has been mathematically proven that under some mild conditions, commonly assumed in the analysis of response data, the conditional covariance of two items, dichotomously or polytomously scored, is positive if the two items are dimensionally homogeneous and negative…

Physics 30 is a 12th-grade science course for students in Alberta, Canada. This document is the final test for the course. Intended for administration during June 1989, it contains 49 multiple-choice questions and 4 written-response questions. Two-and-one-half hours are allowed for completing the test. No answer key is included since scoring is…

This document outlines the use of machine-scorable open-ended questions for the evaluation of Physics 30 in Alberta. Contents included are: (1) an introduction to the questions; (2) sample instruction sheet; (3) thirty sample items from the complete course; (4) item information including an answer key, difficulty, and source of each item; (5)…

Two issues in the writing of multiple-choice test items were investigated: a comparison of three versus four options; and the use of the inclusive "none of these" option versus a content option. Subjects were 220 introductory psychology students, who were enrolled at a large southwestern university, responding to a final examination in psychology…

A plausible "s"-factor solution for many types of psychological and educational tests is one in which there is one general factor and "s - 1" group- or method-related factors. The bi-factor solution results from the constraint that each item has a non-zero loading on the primary dimension "alpha(sub j1)" and at most…

Testing 1-2-3 is a general purpose testing system developed at the Computer-Based Education Research Laboratory at the University of Illinois for use on NovaNET computer-based education systems. The testing system can be used for: short, teacher-made quizzes, individualized examinations, computer managed instruction curriculum testing,…

To assist instructors in implementing Missouri's Vocational Instructional Management System into the Food Production, Management, and Services Programs, this guide sets forth the competencies identified and validated by occupational food service instructors and personnel from the food service industry. A minimum of two performance objectives per…

Test validity is a concept that has often been ignored in the context of latent trait models and in modern test theory, particularly as it relates to computerized adaptive testing. Some considerations about the validity of a test and of a single item are proposed. This paper focuses on measures that are population-free and that will provide local…

An analysis-of-covariance methodology was used to investigate whether there were population differences between tryout and operational Rasch item b-values relative to differences between pairs of item response theory (IRT) b-values from consecutive operational item administrations. This methodology allowed the evaluation of whether any such…

A target partition analysis (TPA) was used to help seven subject matter experts, with little expertise in testing, evaluate relationships among test items. The subjects were experts in Income Maintenance in the field of public assistance work; they were trainers affiliated with the Professional Development Program's Income Maintenance Training…

A new approach to polychotomous scoring of test items, similar to "max-alpha" scaling (MAS) and known as polyweighting, has been developed. Unlike MAS, this new method of polychotomous scoring provides scoring weights for a given item that are independent of the difficulty of other items in the analysis. Moreover, the scoring weights are…

This study empirically examined the proposition that different item selection procedures contribute to the development of achievement tests that optimally serve different purposes. A test of 110 addition and subtraction problems was given to a sample of 371 first-grade students. Three subsets of 30 items were selected from this 110-item test. The…

This manual provides assistance in developing and editing questions for the Medical Library Association General and Area of Concentration Examinations for Certification or Registration. After describing the certification procedures and the examination itself, suggestions for test item writers are provided, including preparing the item-as-a-whole,…