Chalmers recently published a critique of the use of ordinal a[alpha] proposed in Zumbo et al. as a measure of test reliability in certain research settings. In this response, we take up the task of refuting Chalmers' critique. We identify three broad misconceptions that characterize Chalmers' criticisms: (1) confusing assumptions with…

This monograph provides an epistemological rational for the Accumulative Manifold Validation Analysis [also referred by the acronym "AMOVA"] statistical methodology designed to test psychometric instruments. This form of inquiry is a form of mathematical optimization in the discipline of linear stochastic modelling. AMOVA is an in-depth…

The kappamax reliability index of domain-referenced tests is defined as the upper bound of kappa when all possibile cutoff scores are considered. Computational procedures for kappamax are described, as well as its approximation for long tests, based on Kuder-Richardson formula 21. The sampling error of kappamax, and the effects of test length and…

Huynh (1976) suggested a method of approximating the reliability coefficient of a mastery test. The present study examines the accuracy of Huynh's approximation and also describes a computationally simpler approximation which appears to be generally more accurate than the former. (Author/RL)

A number of different definitions and indices of reliability for mastery tests have recently been proposed in an attempt to cope with possible lack of score variability that attenuates traditional coefficients. One promising index that has been suggested is the proportion of students in a group that are consistently assigned to the same mastery…

A procedure for estimating the reliability of a factorially complex composite is considered. An application of its use with Scholastic Aptitude Test data is provided. (Author/JKS)

A two-stage testing procedure, a routing test followed by one of several alternative second-stage tests, is studied in the situation where the purpose is measurement, not classification. Models are developed, examined, and compared with conventional tests and up-and-down procedures. (DG)

Three techniques for estimating Kuder Richardson reliability (KR20) coefficients for incomplete data are contrasted. The methods are: (1) Henderson's Method 1 (analysis of variance, or ANOVA); (2) Henderson's Method 3 (FITCO); and (3) Koch's method of symmetric sums (SYSUM). A Monte Carlo simulation was used to assess the precision of the three…

The stepped-up reliability coefficient does not have the same standard error as an ordinary correlation coefficient. Fisher's Z -transformation should not be applied to it. Appropriate procedures are suggested. (Author)

Results of a comparative study of F and Q tests, in a randomized block design with one replication per cell, are presented. In addition to these two procedures, a multivariate test was also considered. The model and test statistics, data generation and parameter selection, results, summary and conclusions are presented. Ten tables contain the…

Using the concepts of conditional probability, conditional expectation, and conditional independence, the main results of the classical test theory model can be derived in a very few steps with minimal assumptions. The present effort explores the possibility that present classical test theories can be further condensed. (Author/RC)

A least-squares solution for the method of paired comparisons is given. The approach provokes a theorem regarding the amount of data necessary and sufficient for a solution to be obtained. A measure of the internal consistency of the least-squares fit is developed. (Author/CTM)

Joreskog's model of cogeneric tests is used to analyze agreement between raters. Raters are treated as measuring instruments. The model of cogeneric tests, of which classical parallelism and tau-equivalence are shown to be special cases, is applied to teachers' ratings of students' responses on open-end questions. (Author/RC)

An important determinant of equating process efficiency is the correlation between the anchor test and components of each form. Use of some monotonic function of this correlation as a measure of equating efficiency is suggested. A model relating anchor test length and test reliability to this measure of efficiency is presented. (Author/DWH)