When it is required to estimate item parameters of a large item bank, Multiple Matrix Sampling (MMS) design provides an efficient way while minimizing the test burden on students. The current study exemplifies how to calibrate a large item pool using MMS design for various purposes, such as developing a CAT administration. The purpose of the…

Item banks are generally considered the basis of a new generation of educational measurement. In combination with specialized software, they can facilitate the computerized assembling of multiple pre-equated test forms. However, for advantages of item banks to become fully realized it is important that the item banks store a relatively large…

Successful testing programs rely on high-quality test items to produce reliable scores and defensible exams. However, determining what statistical screening criteria are most appropriate to support these goals can be daunting. This study describes and demonstrates cost-benefit analysis as an empirical approach to determining appropriate screening…

This article reflects on the history of interest assessment in Australia in the last 45 years. In it the author would like to review some aspects of the history of interest assessment in Australia from his personal perspective as a user and researcher. He suggests that the present state of interest assessment in Australia using inventories is…

Shoemaker's conclusions related to the influence of various data base characteristics (reliability, variability of item difficulty indices, and degree of skewness in the normative distribution) on the standard error of a mean estimated via multiple matrix sampling procedures are examined. (Author/RC)

This investigation had two major purposes. The first was to explore the use of an inferential technique called Tukey's Jackknife in establishing a confidence interval about cooefficient alpha reliability. The second purpose was to study the robustness of the Feldt and the jackknife procedures when the data fails to satisfy usual normality…

Many academic tests (e.g. short-answer and multiple-choice) sample required knowledge with questions scoring 0 or 1 (dichotomous scoring). Few textbooks give useful guidance on the length of test needed to do this reliably. Posey's binomial error model of 1932 provides the best starting point, but allows neither for heterogeneity of question…

A rederivation of Lord's formula for estimating variance in multiple matrix sampling is presented as well as the ways Cronbach's coefficient alpha and the Spearman-Brown prophecy formula are related in this context. (Author/JKS)

This study demonstrates that multiple matrix sampling procedures can be used to collect assessment data efficiently, unabstrusively, and reliably. (MB)

An investigation is conducted which presents extensive Monte Carlo results which indicate the conditions under which a procedure using the F distribution can be used to study the robustness of the confidence interval procedures for small samples. A review of the literature is presented. Procedure uses a binary data matrix. Results indicate that…

Test reliability is a concept central to classical test theory and it is commonly stated as a requirement that a test attain a certain level of reliability before it be considered of sufficient quality for practical use. This article discusses the role of reliability in item response theory, and in particular the role of reliability in contexts…

This research examined 2 elements of multiple-choice test construction, balancing the key and optimal number of options. In Experiment 1 the 3 conditions included a balanced key, overrepresentation of a and b responses, and overrepresentation of c and d responses. The results showed that error-patterns were independent of the key, reflecting…

Pennsylvania's Educational Quality Assessment Program provides each participating school with a building level report in which state percentiles are a prominent part. Multiple matrix sampling was being considered as a technique to reduce testing time. However, there was great concern that the error associated with estimating the school mean might…