Publication Date
In 2025 | 0 |
Since 2024 | 0 |
Since 2021 (last 5 years) | 1 |
Since 2016 (last 10 years) | 4 |
Since 2006 (last 20 years) | 5 |
Descriptor
Generalizability Theory | 13 |
Performance Based Assessment | 13 |
Reliability | 13 |
Scores | 7 |
Error of Measurement | 6 |
Evaluation Methods | 3 |
Language Tests | 3 |
Psychometrics | 3 |
Scoring | 3 |
Validity | 3 |
Elementary School Students | 2 |
More ▼ |
Source
Applied Measurement in… | 2 |
Educational and Psychological… | 2 |
Applied Psychological… | 1 |
Journal of Educational… | 1 |
Language Assessment Quarterly | 1 |
Language Testing | 1 |
Psychometrika | 1 |
Author
Brennan, Robert L. | 3 |
Cronbach, Lee J. | 1 |
Edelman, Amanda | 1 |
Gao, Xiaohong | 1 |
Han, Chao | 1 |
Harnisch, Delwyn L. | 1 |
Jiang, Ying Hong | 1 |
Kloser, Matt | 1 |
Lin, Chih-Kai | 1 |
Martínez, José Felipe | 1 |
Miller, M. David | 1 |
More ▼ |
Publication Type
Reports - Research | 10 |
Journal Articles | 9 |
Reports - Evaluative | 2 |
Speeches/Meeting Papers | 2 |
Information Analyses | 1 |
Numerical/Quantitative Data | 1 |
Tests/Questionnaires | 1 |
Education Level
Higher Education | 2 |
Junior High Schools | 1 |
Middle Schools | 1 |
Postsecondary Education | 1 |
Secondary Education | 1 |
Audience
Laws, Policies, & Programs
Assessments and Surveys
What Works Clearinghouse Rating
Martínez, José Felipe; Kloser, Matt; Srinivasan, Jayashri; Stecher, Brian; Edelman, Amanda – Educational and Psychological Measurement, 2022
Adoption of new instructional standards in science demands high-quality information about classroom practice. Teacher portfolios can be used to assess instructional practice and support teacher self-reflection anchored in authentic evidence from classrooms. This study investigated a new type of electronic portfolio tool that allows efficient…
Descriptors: Science Instruction, Academic Standards, Instructional Innovation, Electronic Publishing
Lin, Chih-Kai – Language Testing, 2017
Sparse-rated data are common in operational performance-based language tests, as an inevitable result of assigning examinee responses to a fraction of available raters. The current study investigates the precision of two generalizability-theory methods (i.e., the rating method and the subdividing method) specifically designed to accommodate the…
Descriptors: Data Analysis, Language Tests, Generalizability Theory, Accuracy
Schmidgall, Jonathan – Applied Measurement in Education, 2017
This study utilizes an argument-based approach to validation to examine the implications of reliability in order to further differentiate the concepts of score and decision consistency. In a methodological example, the framework of generalizability theory was used to estimate appropriate indices of score consistency and evaluations of the…
Descriptors: Scores, Reliability, Validity, Generalizability Theory
Han, Chao – Language Assessment Quarterly, 2016
As a property of test scores, reliability/dependability constitutes an important psychometric consideration, and it underpins the validity of measurement results. A review of interpreter certification performance tests (ICPTs) reveals that (a) although reliability/dependability checking has been recognized as an important concern, its theoretical…
Descriptors: Foreign Countries, Scores, English, Chinese
Shin, Yongyun; Raudenbush, Stephen W. – Psychometrika, 2012
Social scientists are frequently interested in assessing the qualities of social settings such as classrooms, schools, neighborhoods, or day care centers. The most common procedure requires observers to rate social interactions within these settings on multiple items and then to combine the item responses to obtain a summary measure of setting…
Descriptors: Generalizability Theory, Neighborhoods, Intervals, Child Care Centers

Brennan, Robert L. – Applied Psychological Measurement, 2000
Reviews relevant aspects of generalizability theory related to performance assessments and discusses the role of various facets in assessing the generalizability of performance assessments. Also considers some popular estimates of reliability for performance assessments from the perspective of generalizability theory. (SLD)
Descriptors: Estimation (Mathematics), Evaluation Methods, Generalizability Theory, Performance Based Assessment

Gao, Xiaohong; Brennan, Robert L. – Applied Measurement in Education, 2001
Studied the sampling variability of estimated variance components using data collected over several years for a listening and writing performance assessment and evaluated the stability of estimated measurement precision. Results indicate that the estimated variance components varied from one year to another and suggest that the measurement…
Descriptors: Estimation (Mathematics), Generalizability Theory, Listening Comprehension Tests, Performance Based Assessment
Brennan, Robert L. – 1993
Not infrequently, investigators assume that reliability for groups is greater than reliability for persons, or that the error variance for groups is less than that for persons. Using generalizability theory, it is shown that this "conventional wisdom" is not necessarily true. Examples are provided from the course-evaluation and the…
Descriptors: Comparative Analysis, Course Evaluation, Generalizability Theory, Measurement Techniques
Miller, M. David – 2002
In 1994 the State Collaborative on Assessment and Student Standards of the Council of Chief State School Officers began a study to examine the generalizability of performance-based assessments (PBAs) for state-mandated assessment programs. The intent was to examine the major sources of error associated with PBAs and the generalizability and…
Descriptors: Elementary Secondary Education, Error of Measurement, Generalizability Theory, Performance Based Assessment
Jiang, Ying Hong; Smith, Philip L. – 2000
With a construct-centered reliability analytical approach the reliability analysis should crystallize the multi-traits or constructs that the test specialists developed to measure from student performance and then estimate the degree of fit between the theoretical expectations of test developers and the performance exhibited by students. This…
Descriptors: Cognitive Tests, Construct Validity, Elementary Education, Elementary School Students

Cronbach, Lee J.; And Others – Educational and Psychological Measurement, 1997
Through the standard error, rather than a reliability coefficient, generalizability theory provides an indicator of the uncertainty attached to school and individual scores on performance assessments. Recommendations are made to apply generalizability theory to current performance assessments, emphasizing practices that differ from usual…
Descriptors: Academic Achievement, Error of Measurement, Generalizability Theory, Performance Based Assessment
Suzuki, Kyoko; Harnisch, Delwyn L. – 1996
This study examined the generalizability and dependability of a performance-based assessment in algebra. Four forms of a five-item test were constructed using different subsets of eight items based on attributes from task analysis. Subjects included 142 "algebra 2" students from 2 high schools in the Midwestern United States and 148 11th…
Descriptors: Algebra, Foreign Countries, Generalizability Theory, High School Students

Ruiz-Primo, Maria Araceli; And Others – Journal of Educational Measurement, 1993
The stability of scores on 2 types of performance assessments, an observed hands-on investigation and a notebook surrogate, was investigated for 29 sixth graders on 2 occasions. Results indicate that student performance and procedures changed and that generalizability across occasions was moderate. Implications for assessment are discussed. (SLD)
Descriptors: Educational Assessment, Elementary School Students, Error of Measurement, Generalizability Theory