A computer program which creates and stores a cumulative item covariance matrix upon each administration of an instrument is described. Use of this program would facilitate keeping a constant log on reliability in situations in which the test is administered to different groups over a period of time. (Author/JKS)

In mastery testing, the raw agreement index and the kappa index may be estimated via one test administration when the test scores follow beta-binomial distributions. This paper reports formulae, tables, and a computer program which facilitate the computation of the standard errors of the estimates. (Author/CTM)

A statistical method for testing the agreement between a judge's assessment of an object or subject and a known standard is developed and shown to be superior to two other methods which appear in the literature. (Author/JKS)

Monte Carlo research with TAILOR, a program using implied orders as a basis for tailored testing, is reported. TAILOR typically required about half the available items to estimate, for each simulated examinee, the responses on the remainder. (Author/CTM)

A new method was developed to assist in the selection of a test length by utilizing computer simulation procedures and item response theory. A demonstration of the method presents results which address the influences of item pool heterogeneity matched to the objectives of interest and the method of item selection. (Author/PN)

Procedures for computing content validity and consistency reliability coefficients and determining the statistical significance of these coefficients are described. Procedures employing the multinomial probability distribution for small samples and normal curve probability estimates for large samples, can be used where judgments are made on…

This paper elaborates on the work of Cardinet, and others, by clarifying some points regarding calculations, specifically with reference to existing computer programs, and by presenting illustrative examples of the calculation and interpretation of several generalizability coefficients from a complex six-facet (factor) design. (Author/RL)

Believing that examinee-by-item interaction should be conceptualized as true score variability rather than as a result of errors of measurement, Lu proposed a modification of Hoyt's analysis of variance reliability procedure. Via a computer simulation study, it is shown that Lu's approach does not separate interaction from error. (Author/RL)

Computerized analysis of multiple choice test items is explained. Examples of item analysis applications in the introductory economics course are discussed with respect to three objectives: to evaluate learning; to improve test items; and to help improve classroom instruction. Problems, costs and benefits of the procedures are identified. (JMD)