This paper describes and compares procedures for estimating the reliability of proficiency tests that are scored with latent structure models. Results suggest that the predictive estimate is the most accurate of the procedures. (Author/BW)

A more general method for calculating the Kappa measure of nominal rating agreement among multiple raters is presented. It can be used across a broad range of rating designs, including those in which raters vary with respect to their base rates and how many subjects they rate in common. (Author/BW)

Expressions involving optimal sign vectors are derived so as to yield two new applications. First, coefficient alpha for the sign-weighted composite is maximized in analogy to Lord's scale-independent solution with differential weights. Second, optimal sign vectors are used to define two groups of objects that are maximally distinct. (Author/CTM)

A coefficient of inter-rater agreement is presented which describes the magnitude of observer agreement as the probability estimated under a quasi-independence model that responses from different observers will be in agreement. (Author/JKS)

Guilford's average interrater correlation coefficient is shown to be related to the Friedman Rank Sum statistic. Under the null hypothesis of zero correlation, the resultant distribution is known and the hypothesis can be tested. Large sample and tied score cases are also considered. An example from Guilford (1954) is presented. (Author)

The robustness of hypothesis tests for the correlation coefficient under varying conditions is discussed. The effects of violations of the assumptions of linearity, homoscedasticity, and kurtosis are examined. (JKS)

Evaluation of information found on the Internet requires the same assessment of reliability, credibility, perspective, purpose and author credentials as required with print materials. Things to check include whether the source is from a moderated or unmoderated list or FTP (file transfer protocol) site; directories for affiliation and biographical…

Conducted a reliability generalization study for the State-Trait Anxiety Inventory (C. Spielberger, 1983) by reviewing and classifying 816 research articles. Average reliability coefficients were acceptable for both internal consistency and test-retest reliability, but variation was present among the estimates. Other differences are discussed.…

Develops formulas to cope with the situation in which the reliability of test scores must be approximated even though no examinee has taken the complete instrument. Develops different estimators for part tests that are judged to be classically parallel, tau-equivalent, or congeneric. Proposes standards for differentiating among these three models.…

A formally simple expression for the maximal reliability of a linear composite is provided. Its theoretical implications and its relation to existing results for reliability are discussed. (Author/SLD)

Studied the use of descriptive graphic rating scales by 11 raters to evaluate students' work, exploring different design features. Used a Rasch-model based rating scale analysis to determine that all the continuous scales could be considered to have at least five points, and that defined midpoints did not result in higher student separation…