This resource has been written specifically for higher education practitioners who are interested in improving students' learning experiences through the process of researching their own practice. We use the term 'higher education practitioners' to describe all who work in universities and who have a stake in students' learning experiences.…

Reviews some of the more salient conceptual features of generalizability (G) theory. Provides a framework for considering some misconceptions about G theory. Focuses on particularly prevalent or serious misconceptions. G theory provides a rich conceptual framework and tools for working with a defined universe, but it does not tell an investigator…

The authors center their presentation of material around a conceptual understanding of psychometric issues, such as validity and reliability, and on purpose rather than procedure, the "why" rather than the "how to." Their goal is to introduce psychometric principles at a level that is deeper and more focused than found in introductory…

Generalizability theory is described as a way to provide a unified framework for examining the dependability of measurements in the health sciences. Generalizability theory is appropriate whether the measurement is norm- or criterion-referenced, an individual or aggregate-level variable, and it is useful when there are single or multiple sources…

Presents a model-based approach to rater reliability for essays read by multiple raters. The approach is motivated by generalizability theory, and variation of rater severity and rater inconsistency is considered in the presence of between-examinee variations. Illustrates methods with data from standardized educational tests. (Author/SLD)

This paper presents a review of the literature on reliability in qualitative studies. Reliability is defined as the extent to which studies can be replicated, using the same methods, and getting the same results. It is the degree to which data are independent of the accidental circumstances of the research. The review includes the following three…

Results from a performance assessment in which 68 high school students wrote essays support the use of latent variable modeling for estimating reliability, concurrent validity, and generalizability of a scoring rubric. The latent variable modeling approach overcomes the limitations of certain conventional statistical techniques in handling…