Publication Date
In 2025 | 0 |
Since 2024 | 0 |
Since 2021 (last 5 years) | 3 |
Since 2016 (last 10 years) | 13 |
Since 2006 (last 20 years) | 22 |
Descriptor
Generalizability Theory | 39 |
Interrater Reliability | 39 |
Test Reliability | 39 |
Scores | 13 |
Foreign Countries | 9 |
Scoring | 9 |
Test Validity | 9 |
Evaluation Methods | 8 |
Error of Measurement | 7 |
Performance Based Assessment | 7 |
Writing Evaluation | 7 |
More ▼ |
Source
Author
Aktas, Mehtap | 2 |
Atilgan, Hakan | 2 |
Capie, William | 2 |
Johnson, Evelyn S. | 2 |
Uzun, N. Bilge | 2 |
Abedi, Jamal | 1 |
Aksu, Gökhan | 1 |
Alici, Devrim | 1 |
Alkhuwaiter, Munirah | 1 |
Alves, Cecilia Brito | 1 |
Anderson, Judith A. | 1 |
More ▼ |
Publication Type
Journal Articles | 26 |
Reports - Research | 26 |
Reports - Evaluative | 11 |
Speeches/Meeting Papers | 11 |
Reports - Descriptive | 2 |
Tests/Questionnaires | 1 |
Education Level
Audience
Researchers | 5 |
Laws, Policies, & Programs
Assessments and Surveys
Teacher Performance… | 2 |
Medical College Admission Test | 1 |
Students Evaluation of… | 1 |
What Works Clearinghouse Rating
Comparison of the Results of the Generalizability Theory with the Inter-Rater Agreement Coefficients
Eser, Mehmet Taha; Aksu, Gökhan – International Journal of Curriculum and Instruction, 2022
The agreement between raters is examined within the scope of the concept of "inter-rater reliability". Although there are clear definitions of the concepts of agreement between raters and reliability between raters, there is no clear information about the conditions under which agreement and reliability level methods are appropriate to…
Descriptors: Generalizability Theory, Interrater Reliability, Evaluation Methods, Test Theory
Atilgan, Hakan; Demir, Elif Kübra; Ogretmen, Tuncay; Basokcu, Tahsin Oguz – International Journal of Progressive Education, 2020
It has become a critical question what the reliability level would be when open-ended questions are used in large-scale selection tests. One of the aims of the present study is to determine what the reliability would be in the event that the answers given by test-takers are scored by experts when open-ended short answer questions are used in…
Descriptors: Foreign Countries, Secondary School Students, Test Items, Test Reliability
Clain, Alex E.; Alkhuwaiter, Munirah; Davidson, Kate; Martin-Harris, Bonnie – Journal of Speech, Language, and Hearing Research, 2022
Purpose: The purpose of this study was to extend the assessment of the psychometric properties of the Modified Barium Swallow Impairment Profile (MBSImP). Here, we re-examined structural validity and internal consistency using a large clinical-registry data set and formally examined rater reliability in a smaller data set. Method: This study…
Descriptors: Diagnostic Tests, Disability Identification, Physical Disabilities, Eating Disorders
Atilgan, Hakan – Eurasian Journal of Educational Research, 2019
Purpose: This study intended to examine the generalizability and reliability of essay ratings within the scope of the generalizability (G) theory. Specifically, the effect of raters on the generalizability and reliability of students' essay ratings was examined. Furthermore, variations of the generalizability and reliability coefficients with…
Descriptors: Foreign Countries, Essay Tests, Test Reliability, Interrater Reliability
D'Agostino, Jerome V.; Rodgers, Emily; Winkler, Christa; Johnson, Tracy; Berenbon, Rebecca – Reading Psychology, 2021
Running Records provide a standardized method for recording and assessing students' oral reading behaviors and are excellent formative assessment tools to guide instructional decision-making. This study expands on prior Running Record reliability work by evaluating the extent to which external raters and teachers consistently assessed students'…
Descriptors: Accuracy, Oral Reading, Generalizability Theory, Error Correction
Roduta Roberts, Mary; Alves, Cecilia Brito; Werther, Karin; Bahry, Louise M. – Journal of Psychoeducational Assessment, 2019
The purpose of this study was to examine the reliability and sources of score variation from a performance assessment of practice competencies within an occupational therapy program. Data from 99 students who participated in a practical exam were examined. A generalizability analysis of analytic, total, and overall holistic scores was completed…
Descriptors: Performance Based Assessment, Test Reliability, Scores, Occupational Therapy
Uzun, N. Bilge; Alici, Devrim; Aktas, Mehtap – European Journal of Educational Research, 2019
The purpose of study is to examine the reliability of analytical rubrics and checklists developed for the assessment of story writing skills by means of generalizability theory. The study group consisted of 52 students attending the 5th grade at primary school and 20 raters in Mersin University. The G study was carried out with the fully crossed…
Descriptors: Foreign Countries, Scoring Rubrics, Check Lists, Writing Tests
Irby, Sarah M.; Floyd, Randy G. – Psychology in the Schools, 2017
This study examined the exchangeability of total scores (i.e., intelligent quotients [IQs]) from three brief intelligence tests. Tests were administered to 36 children with intellectual giftedness, scored live by one set of primary examiners and later scored by a secondary examiner. For each student, six IQs were calculated, and all 216 values…
Descriptors: Intelligence Tests, Gifted, Error of Measurement, Scores
Uzun, N. Bilge; Aktas, Mehtap; Asiret, Semih; Yormaz, Seha – Asian Journal of Education and Training, 2018
The goal of this study is to determine the reliability of the performance points of dentistry students regarding communication skills and to examine the scoring reliability by generalizability theory in balanced random and fixed facet (mixed design) data, considering also the interactions of student, rater and duty. The study group of the research…
Descriptors: Foreign Countries, Generalizability Theory, Scores, Test Reliability
Cankoy, Osman; Özder, Hasan – EURASIA Journal of Mathematics, Science & Technology Education, 2017
The aim of this study is to develop a scoring rubric to assess primary school students' problem posing skills. The rubric including five dimensions namely solvability, reasonability, mathematical structure, context and language was used. The raters scored the students' problem posing skills both with and without the scoring rubric to test the…
Descriptors: Generalizability Theory, Elementary School Students, Foreign Countries, Problem Solving
Charalambous, Charalambos Y.; Kyriakides, Ermis; Tsangaridou, Niki; Kyriakides, Leonidas – School Effectiveness and School Improvement, 2017
Heightened accountability pressures and an increased emphasis on teaching quality have directed scholarly attention to scrutinizing instruction, particularly with respect to issues of validity and reliability. However, these attempts have largely been directed toward "core" content areas and investigated generic or content-specific…
Descriptors: Physical Education, Instructional Effectiveness, Lesson Plans, Interrater Reliability
Oh, Yoonkyung; Osgood, D. Wayne; Smith, Emilie P. – Journal of Early Adolescence, 2015
The importance of afterschool hours for youth development is widely acknowledged, and afterschool settings have recently received increasing attention as an important venue for youth interventions, bringing a growing need for reliable and valid measures of afterschool quality. This study examined the extent to which the two observational tools,…
Descriptors: After School Programs, Program Effectiveness, Observation, Rating Scales
Crawford, Angela R.; Johnson, Evelyn S.; Moylan, Laura A.; Zheng, Yuzhu – Grantee Submission, 2018
This study describes the development and initial psychometric evaluation of a Recognizing Effective Special Education Teachers (RESET) teacher observation instrument. Specifically, the study uses generalizability theory to compare two versions of a rubric, one with general descriptors of performance levels and one with item-specific descriptors of…
Descriptors: Special Education Teachers, Direct Instruction, Observation, Teaching Methods
Uto, Masaki; Ueno, Maomi – IEEE Transactions on Learning Technologies, 2016
As an assessment method based on a constructivist approach, peer assessment has become popular in recent years. However, in peer assessment, a problem remains that reliability depends on the rater characteristics. For this reason, some item response models that incorporate rater parameters have been proposed. Those models are expected to improve…
Descriptors: Item Response Theory, Peer Evaluation, Bayesian Statistics, Simulation
Moser, Gary P.; Sudweeks, Richard R.; Morrison, Timothy G.; Wilcox, Brad – Reading Psychology, 2014
This study examined ratings of fourth graders' oral reading expression. Randomly assigned participants (n = 36) practiced repeated readings using narrative or informational passages for 7 weeks. After this period raters used the "Multidimensional Fluency Scale" (MFS) on two separate occasions to rate students' expressive…
Descriptors: Elementary School Students, Oral Reading, Reading Skills, Suprasegmentals