We derive an analytic model of the inter-judge correlation as a function of five underlying parameters. Inter-cue correlation and the number of cues capture our assumptions about the environment, while differentiations between cues, the weights attached to the cues, and (un)reliability describe assumptions about the judges. We study the relative…

The problem of objectively comparing two independently determined partitions of N objects or variables is discussed. A similarity measure based on the simple matching coefficient is defined and related to previously suggested measures. (Author/JKS)

A proportional reduction in loss (PRL) measure for reliability of categorical data is explored for the situation in which each of "N" judges assigns a subject to one of "K" categories. Calculating a lower bound for reliability under more general conditions than had been proposed is demonstrated. (SLD)

A previous paper proposed a generalized family of coefficients of relational agreement for multiple judges. It focused on the concept of empirically meaningful relationships. This paper presents an ordinal coefficient of relational agreement as a special case of the generalized family. The proposed ordinal coefficient encompasses other ordinal…

A statistical method for testing the agreement between a judge's assessment of an object or subject and a known standard is developed and shown to be superior to two other methods which appear in the literature. (Author/JKS)