NotesFAQContact Us
Collection
Advanced
Search Tips
Showing all 7 results Save | Export
A Ranking Method for Evaluating Constructed Responses
Peer reviewed Peer reviewed
Direct linkDirect link
Attali, Yigal – Educational and Psychological Measurement, 2014
This article presents a comparative judgment approach for holistically scored constructed response tasks. In this approach, the grader rank orders (rather than rate) the quality of a small set of responses. A prior automated evaluation of responses guides both set formation and scaling of rankings. Sets are formed to have similar prior scores and…
Descriptors: Responses, Item Response Theory, Scores, Rating Scales
The Oceanography Concept Inventory: A Semicustomizable Assessment for Measuring Student Understanding of Oceanography
Peer reviewed Peer reviewed
PDF on ERIC Download full text
Direct linkDirect link
Arthurs, Leilani; Hsia, Jennifer F.; Schweinle, William – Journal of Geoscience Education, 2015
We developed and evaluated an Oceanography Concept Inventory (OCI), which used a mixed-methods approach to test student achievement of 11 learning goals for an introductory-level oceanography course. The OCI was designed with expert input, grounded in research on student (mis)conceptions, written with minimal jargon, tested on 464 students, and…
Descriptors: Oceanography, Mixed Methods Research, Academic Achievement, Introductory Courses
Psychometric Properties of the Multiple Mini-Interview Used for Medical Admissions: Findings from Generalizability and Rasch Analyses
Peer reviewed Peer reviewed
Direct linkDirect link
Sebok, Stefanie S.; Luu, King; Klinger, Don A. – Advances in Health Sciences Education, 2014
The multiple mini-interview (MMI) has become an increasingly popular admissions method for selecting prospective students into professional programs (e.g., medical school). The MMI uses a series of short, labour intensive simulation stations and scenario interviews to more effectively assess applicants' non-cognitive qualities such as…
Descriptors: Medical Education, Medical Students, College Admission, Generalizability Theory
A Comparison of the Approaches of Generalizability Theory and Item Response Theory in Estimating the Reliability of Test Scores for Testlet-Composed Tests
Peer reviewed Peer reviewed
Direct linkDirect link
Lee, Guemin; Park, In-Yong – Asia Pacific Education Review, 2012
Previous assessments of the reliability of test scores for testlet-composed tests have indicated that item-based estimation methods overestimate reliability. This study was designed to address issues related to the extent to which item-based estimation methods overestimate the reliability of test scores composed of testlets and to compare several…
Descriptors: Generalizability Theory, Simulation, Computation, Item Response Theory
Measurement of Classroom Teaching Quality with Item Response Theory
Peer reviewed Peer reviewed
PDF on ERIC Download full text
Kelcey, Ben; McGinn, Daniel; Hill, Heather – Society for Research on Educational Effectiveness, 2013
Recent policy has charged schools and districts with maintaining highly qualified teachers and differentiating among teachers in terms of their effectiveness (U.S. Department of Education, 2009). This emphasis has driven the development and implementation of teacher quality measures which are increasingly being used to evaluate teachers with…
Descriptors: Teacher Effectiveness, Measures (Individuals), Observation, Teacher Evaluation
Examining Reliability and Validity of Job Analysis Survey Data.
PDF on ERIC Download full text
Wang, Ning; Wiser, Randall F.; Newman, Larry S. – 1999
Job analysis has played a fundamental role in developing and validating licensure and certification examinations, but research on what constitutes reliable and valid job analysis data is lacking. This paper examines the reliability and validity of job analysis survey results. Generalizability theory and the multi-facet Rasch item response theory…
Descriptors: Generalizability Theory, Goodness of Fit, Item Response Theory, Job Analysis
Score Reliability of a Test Composed of Passage-Based Testlets: A Generalizability Theory Perspective.
Lee, Yong-Won – 2002
The purpose of this study was to investigate the impact of local item dependence (LID) in passage-based testlets on the test score reliability of an English as a Foreign Language (EFL) reading comprehension test from the perspective of generalizability (G) theory. Definitions and causes of LID in passage-based testlets are reviewed within the…
Descriptors: English (Second Language), Foreign Countries, Generalizability Theory, High School Students