Successful testing programs rely on high-quality test items to produce reliable scores and defensible exams. However, determining what statistical screening criteria are most appropriate to support these goals can be daunting. This study describes and demonstrates cost-benefit analysis as an empirical approach to determining appropriate screening…

Shoemaker's conclusions related to the influence of various data base characteristics (reliability, variability of item difficulty indices, and degree of skewness in the normative distribution) on the standard error of a mean estimated via multiple matrix sampling procedures are examined. (Author/RC)

This investigation had two major purposes. The first was to explore the use of an inferential technique called Tukey's Jackknife in establishing a confidence interval about cooefficient alpha reliability. The second purpose was to study the robustness of the Feldt and the jackknife procedures when the data fails to satisfy usual normality…

An investigation is conducted which presents extensive Monte Carlo results which indicate the conditions under which a procedure using the F distribution can be used to study the robustness of the confidence interval procedures for small samples. A review of the literature is presented. Procedure uses a binary data matrix. Results indicate that…

Achievement tests which are specifically linked to an instructional program and have been developed in relation to an objectives base and/or to an item generation rule are considered, as well as student response data. Three types of studies are outlined and the kind of procedures thought useful illustrated. As various methods for examining…

Described and listed herein with concomitant sample input and output is the Fortran IV program which estimates parameters and standard errors of estimate per parameters for parameters estimated through multiple matrix sampling. The specific program is an improved and expanded version of an earlier version. (Author/BJG)

The literature on criterion referenced testing is full of discussions concerning whether classical measurement techniques are appropriate, whether variance is necessary, whether new indices of reliability are needed, and the like. What appears to be lacking, however, is a clear and simple discussion of why the problems occur. This paper suggests…

A set of Tests of Scientific Thinking were developed for possible use as criterion measures in research on creativity. Scores on the tests describe both quality and quantity of ideas produced in formulating hypotheses, evaluating proposals, solving methodological problems, and devising methods for measuring constructs. (Author/CTM)

Multiple matrix sampling is a sampling design in which both test items and examinees are randomly sampled from their respective populations. This study was designed to develop and assess a method for computing an estimate of a correlation coefficient when a multiple matrix sampling design is used. The examinee populations included 212 third-grade…

The concept under investigation was the reliability of estimates of mean scores of groups under various assumptions of multiple-matrix sampling when reliabilities are computed according to procedures based on generalizability theory. Four different cases were compared with respect to the generalizability coefficients depending upon whether pupils…

The implications of a mathematical model of test scores are explored where the data are limited to a random sample of items without replacement from an indefinitely large population or item domain in which items are scored either zero or one. The purpose is to obtain an unbiased estimate of a student's proportion of items correct in the item…

A central problem for the user of domain-referenced tests in instruction is deciding who has passed and who has failed. Two procedures were presented and discussed. The first, employing classical test theory, was found to be more useful for larger domains and where the passing standard is 70 percent or less. The sampling procedure suggested by…