NotesFAQContact Us
Collection
Advanced
Search Tips
Showing all 7 results Save | Export
Peer reviewed Peer reviewed
Direct linkDirect link
DeMars, Christine – Applied Measurement in Education, 2015
In generalizability theory studies in large-scale testing contexts, sometimes a facet is very sparsely crossed with the object of measurement. For example, when assessments are scored by human raters, it may not be practical to have every rater score all students. Sometimes the scoring is systematically designed such that the raters are…
Descriptors: Educational Assessment, Measurement, Data, Generalizability Theory
Peer reviewed Peer reviewed
Direct linkDirect link
Schmidgall, Jonathan – Applied Measurement in Education, 2017
This study utilizes an argument-based approach to validation to examine the implications of reliability in order to further differentiate the concepts of score and decision consistency. In a methodological example, the framework of generalizability theory was used to estimate appropriate indices of score consistency and evaluations of the…
Descriptors: Scores, Reliability, Validity, Generalizability Theory
Peer reviewed Peer reviewed
Direct linkDirect link
Taylor, Melinda Ann; Pastor, Dena A. – Applied Measurement in Education, 2013
Although federal regulations require testing students with severe cognitive disabilities, there is little guidance regarding how technical quality should be established. It is known that challenges exist with documentation of the reliability of scores for alternate assessments. Typical measures of reliability do little in modeling multiple sources…
Descriptors: Generalizability Theory, Alternative Assessment, Test Reliability, Scores
Peer reviewed Peer reviewed
Direct linkDirect link
Shumate, Steven R.; Surles, James; Johnson, Robert L.; Penny, Jim – Applied Measurement in Education, 2007
Increasingly, assessment practitioners use generalizability coefficients to estimate the reliability of scores from performance tasks. Little research, however, examines the relation between the estimation of generalizability coefficients and the number of rubric scale points and score distributions. The purpose of the present research is to…
Descriptors: Generalizability Theory, Monte Carlo Methods, Measures (Individuals), Program Effectiveness
Peer reviewed Peer reviewed
Lee, Guemin; Frisbie, David A. – Applied Measurement in Education, 1999
Studied the appropriateness and implications of using a generalizability theory approach to estimating the reliability of scores from tests composed of testlets. Analyses of data from two national standardization samples suggest that manipulating the number of passages is a more productive way to obtain efficient measurement than manipulating the…
Descriptors: Generalizability Theory, Models, National Surveys, Reliability
Peer reviewed Peer reviewed
Marzano, Robert J. – Applied Measurement in Education, 2002
Two studies, each involving 10 eighth graders,compared the findings from generalizability (G) studies and alternative decision (D) studies for 4 approaches to scoring classroom assessments. In terms of less rater x person variability and higher G and D coefficients, the methods ranked in this order: topic-specific rubric, constrained point,…
Descriptors: Comparative Analysis, Decision Making, Generalizability Theory, Junior High School Students
Peer reviewed Peer reviewed
Fitzpatrick, Anne R.; Lee, Guemin; Gao, Furong – Applied Measurement in Education, 2001
Used generalizability theory to assess the variation in school scores across very short test forms that measured mathematics scores in grades 4 and 8. More than 25,000 students took each form of the 3 tests for each grade. Results demonstrate the lack of comparability in school scores across short, nonparallel tests forms and the importance of…
Descriptors: Comparative Analysis, Elementary School Students, Generalizability Theory, Institutional Characteristics