Assuming item parameters on a test are known constants, the reliability coefficient for item response theory (IRT) ability estimates is defined for a population of examinees in two different ways: as (a) the product-moment correlation between ability estimates on two parallel forms of a test and (b) the squared correlation between the true…

We propose a new method of structural equation modeling (SEM) for longitudinal and time series data, named Dynamic GSCA (Generalized Structured Component Analysis). The proposed method extends the original GSCA by incorporating a multivariate autoregressive model to account for the dynamic nature of data taken over time. Dynamic GSCA also…

As pointed out by Sijtsma ("in press"), coefficient alpha is inappropriate as a single summary of the internal consistency of a composite score. Better estimators of internal consistency are available. In addition to those mentioned by Sijtsma, an old dimension-free coefficient and structural equation model based coefficients are…

There are three fundamental problems in Sijtsma ("Psychometrika," 2008): (1) contrary to the name, the glb is not the greatest lower bound of reliability but rather is systematically less than omega[subscript t] (McDonald, "Test theory: A unified treatment," Erlbaum, Hillsdale, 1999), (2) we agree with Sijtsma that when considering how well a test…

The general use of coefficient alpha to assess reliability should be discouraged on a number of grounds. The assumptions underlying coefficient alpha are unlikely to hold in practice, and violation of these assumptions can result in nontrivial negative or positive bias. Structural equation modeling was discussed as an informative process both to…

A method is presented for estimating reliability using structural equation modeling (SEM) that allows for nonlinearity between factors and item scores. Assuming the focus is on consistency of summed item scores, this method for estimating reliability is preferred to those based on linear SEM models and to the most commonly reported estimate of…

We derive an analytic model of the inter-judge correlation as a function of five underlying parameters. Inter-cue correlation and the number of cues capture our assumptions about the environment, while differentiations between cues, the weights attached to the cues, and (un)reliability describe assumptions about the judges. We study the relative…

Social scientists are frequently interested in assessing the qualities of social settings such as classrooms, schools, neighborhoods, or day care centers. The most common procedure requires observers to rate social interactions within these settings on multiple items and then to combine the item responses to obtain a summary measure of setting…

In behavioural sciences, local dependence and DIF are common, and purification procedures that eliminate items with these weaknesses often result in short scales with poor reliability. Graphical loglinear Rasch models (Kreiner & Christensen, in "Statistical Methods for Quality of Life Studies," ed. by M. Mesbah, F.C. Cole & M.T.…

This discussion paper argues that both the use of Cronbach's alpha as a reliability estimate and as a measure of internal consistency suffer from major problems. First, alpha always has a value, which cannot be equal to the test score's reliability given the inter-item covariance matrix and the usual assumptions about measurement error. Second, in…

The critical reactions of Bentler (2009, doi: 10.1007/s11336-008-9100-1), Green and Yang (2009a, doi: 10.1007/s11336-008-9098-4 ; 2009b, doi: 10.1007/s11336-008-9099-3), and Revelle and Zinbarg (2009, doi: 10.1007/s11336-008-9102-z) to Sijtsma's (2009, doi: 10.1007/s11336-008-9101-0) paper on Cronbach's alpha are addressed. The dissemination of…

Reliability captures the influence of error on a measurement and, in the classical setting, is defined as one minus the ratio of the error variance to the total variance. Laenen, Alonso, and Molenberghs ("Psychometrika" 73:443-448, 2007) proposed an axiomatic definition of reliability and introduced the R[subscript T] coefficient, a measure of…

This article is concerned with estimation of components of maximum generalizability in multifacet experimental designs involving multiple dependent measures. An example of a two-facet partially nested design is provided. (Author/RC)

A new measure for reliability of a rating scale is introduced, based on the classical definition of reliability, as the ratio of the true score variance and the total variance. Clinical trial data can be employed to estimate the reliability of the scale in use, whenever repeated measurements are taken. The reliability is estimated from the…

Coefficient Kappa is generally defined in terms of procedures of computation rather than in terms of a population. Here a population definition is proposed. Factors influencing the magnitude of Kappa are identified. Strategies to improve reliability are proposed, including that of combining multiple unreliable diagnoses. (Author/CTM)