The effectiveness of using video replays to judge gymnastics vaulting was examined by determining reliability of judges' scores and comparing mean scores for original (live) vaults and replays. Statistical analyses suggest that variability of scores was greater after the live vault, and that judges' reliability may be enhanced by replays.…

Procedures are developed for testing the hypothesis that Cronbach's alpha reliability coefficient is equal for two tests given to the same subjects. (Author/JKS)

The classical estimate of a binomial probability function is to estimate its mean in the usual manner and to substitute the results in the appropriate expression. Two alternative estimation procedures are described and examined. Emphasis is given to the single administration estimate of the mastery test reliability. (Author/CTM)

Methods for evaluating the consistency of responses to test items were compared. When a researcher is unwilling to make the assumptions of classical test theory, has only a small number of items, or is in a tailored testing context, Cliff's dominance indices may be useful. (Author/CTM)

Generalizability theory is used to characterize and quantify expected variance in cutting scores and to compare the Nedelsky and Angoff procedures for establishing a cutting score. Results suggest that the restricted nature of the Nedelsky (inferred) probability scale may limit its applicability in certain contexts. (Author/BW)

Studies in applied behavior analysis have used two expressions of reliability for human observations: percentage agreement and correlational techniques (including the phi coefficient). Formulas for converting percentage agreement scores to phi coefficients and vice versa are presented. (Author/RL)

Some practical guidelines for developing multiple choice tests are offered. Included are three steps: (1) test design; (2) proper construction of test items; and (3) item analysis and evaluation. (JMF)

A comprehensive overview of the beginnings of attitude measurement is merged with a discussion of recent developments. A synthesis of new research on technical issues related to the reliability and validity of attitude measures and contemporary views on attitude formation and change are presented. (Author)

Examined the test-retest reliability of a rating-scale sociometric technique with four-year-old children. Results showed test-retest reliability over a four-week interval was high compared to the stability of the traditional positive and negative nomination scores. (JMB)

Within the beta-binomial Bayesian framework, procedures are described for the evaluation of the kappa index of reliability on the basis of one administration of a domain-referenced test. Major factors affecting this index include cutoff score, test score variability and test length. Empirical data which substantiate some theoretical trends deduced…

A number of different reliability coefficients have recently been proposed for tests used to differentiate between groups such as masters and nonmasters. One promising index is the proportion of students in a class that are consistently assigned to the same mastery group across two testings. The present paper proposes a single test administration…

Estimates of the internal consistency and reliability of the first principal component are provided through the use of the largest characteristic root and associated vector of the equicorrelation matrix. The estimate of the internal consistency is also shown to be a lower bound for the measure provided by Horn (1969). (Author)

The analysis of variance model for estimating reliability in norm referenced tests is extended to criterion referenced tests. The essential modification is that the criterion or cut-off score is substituted for the population mean. An example and discussion are presented. (JKS)

Dato de Gruijter is correct in the recent conclusion that one equation derived by the present authors should be changed to reflect that it is an approximation, but it is still argued that percentile ranks for difficult tests can have substantially lower reliability and information relative to their number correct scores holds. (SLD)

It has often been asserted that the reliability of a measure places an upper limit on its validity. This article demonstrates in theory that validity can rise when reliability declines, even when validity evidence is a correlation with an acceptable criterion. Whether empirical examples can actually be found is an open question. (SLD)