In adolescence research, the treatment of measurement reliability is often fragmented, and it is not always clear how different reliability coefficients are related. We show that generalizability theory (G-theory) is a comprehensive framework of measurement reliability, encompassing all other reliability methods (e.g., Pearson "r,"…

There are situations within middle school settings where measurements of students and teachers are used for high-stakes decisions. For example, student performance is used as an indicator of teacher quality or determines student eligibility for particular types of support services. Given the high-stakes nature of these types of assessments,…

Student-teacher interactions are dynamic relationships that change and evolve over the course of a school year. Measuring classroom quality through observations that focus on these interactions presents challenges when observations are conducted throughout the school year. Variability in observed scores could reflect true changes in the quality of…

Intraclass correlation coefficients (ICCs) are commonly used in behavioral measurement and psychometrics when a researcher is interested in the relationship among variables of a common class. The formulas for deriving ICCs, or generalizability coefficients, vary depending on which models are specified. This article gives the equations for…

Reviews the criticisms of sampling assumptions in generalizability theory (and in reliability theory) and examines the feasibility of using representative sampling, stratification, homogeneity assumptions, and replications to address these criticisms. Suggests some general outlines for the conduct of generalizability theory studies. (SLD)

The validity generalization model of meta analysis is described and modified for application to concurrent validity studies of motor behavior. An example is presented. (Author/MT)

Generalizability theory is described as a way to provide a unified framework for examining the dependability of measurements in the health sciences. Generalizability theory is appropriate whether the measurement is norm- or criterion-referenced, an individual or aggregate-level variable, and it is useful when there are single or multiple sources…

A structural equation modeling approach to scale reliability evaluation can be employed to estimate generalizability theory indexes in settings where sampling of subjects and conditions is carried out. In one- and two-facet crossed designs, it is demonstrated how this method can be used to obtain estimates of relative generalizability…

Preffi 2.0 is an evidence-based Dutch quality assessment instrument for health promotion interventions. It is mainly intended for both planning and assessing one's own projects but can also be used to assess other people's projects (external use). This article reports a study on the reliability of Preffi as an external quality assessment…

The extent to which a scale score generalizes to a latent variable common to all of the scale's indicators is indexed by the scale's general factor saturation. Seven techniques for estimating this parameter--omega[hierarchical] (omega[subscript h])--are compared in a series of simulated data sets. Primary comparisons were based on 160 artificial…