Publication Date
| In 2025 | 0 |
| Since 2024 | 0 |
| Since 2021 (last 5 years) | 2 |
| Since 2016 (last 10 years) | 3 |
| Since 2006 (last 20 years) | 4 |
Descriptor
| Comparative Analysis | 6 |
| Error of Measurement | 6 |
| Evaluators | 6 |
| Item Response Theory | 4 |
| Scoring | 3 |
| Undergraduate Students | 3 |
| Accuracy | 2 |
| English (Second Language) | 2 |
| Foreign Countries | 2 |
| Reliability | 2 |
| Second Language Learning | 2 |
| More ▼ | |
Source
| Applied Measurement in… | 1 |
| CALICO Journal | 1 |
| Educational and Psychological… | 1 |
| International Journal of… | 1 |
| Psychological Methods | 1 |
Author
| Dekle, Dawn J. | 1 |
| Karakaya, Ismail | 1 |
| Kunnan, Antony John | 1 |
| Lee, Won-Chan | 1 |
| Leung, Denis H. Y. | 1 |
| Linacre, John M. | 1 |
| Liu, Sha | 1 |
| Sata, Mehmet | 1 |
| Song, Yoon Ah | 1 |
| Woehr, David J. | 1 |
| Zhu, Min | 1 |
| More ▼ | |
Publication Type
| Journal Articles | 5 |
| Reports - Research | 5 |
| Reports - Evaluative | 1 |
| Speeches/Meeting Papers | 1 |
Education Level
| Higher Education | 2 |
| Postsecondary Education | 2 |
Audience
| Researchers | 1 |
Laws, Policies, & Programs
Assessments and Surveys
| International English… | 1 |
What Works Clearinghouse Rating
Direct linkSong, Yoon Ah; Lee, Won-Chan – Applied Measurement in Education, 2022
This article presents the performance of item response theory (IRT) models when double ratings are used as item scores over single ratings when rater effects are present. Study 1 examined the influence of the number of ratings on the accuracy of proficiency estimation in the generalized partial credit model (GPCM). Study 2 compared the accuracy of…
Descriptors: Item Response Theory, Item Analysis, Scores, Accuracy
Investigating the Impact of Rater Training on Rater Errors in the Process of Assessing Writing Skill
Download full textSata, Mehmet; Karakaya, Ismail – International Journal of Assessment Tools in Education, 2022
In the process of measuring and assessing high-level cognitive skills, interference of rater errors in measurements brings about a constant concern and low objectivity. The main purpose of this study was to investigate the impact of rater training on rater errors in the process of assessing individual performance. The study was conducted with a…
Descriptors: Evaluators, Training, Comparative Analysis, Academic Language
Liu, Sha; Kunnan, Antony John – CALICO Journal, 2016
This study investigated the application of "WriteToLearn" on Chinese undergraduate English majors' essays in terms of its scoring ability and the accuracy of its error feedback. Participants were 163 second-year English majors from a university located in Sichuan province who wrote 326 essays from two writing prompts. Each paper was…
Descriptors: Foreign Countries, Undergraduate Students, English (Second Language), Second Language Learning
Dekle, Dawn J.; Leung, Denis H. Y.; Zhu, Min – Psychological Methods, 2008
Across many areas of psychology, concordance is commonly used to measure the (intragroup) agreement in ranking a number of items by a group of judges. Sometimes, however, the judges come from multiple groups, and in those situations, the interest is to measure the concordance between groups, under the assumption that there is some within-group…
Descriptors: Item Response Theory, Statistical Analysis, Psychological Studies, Evaluators
Linacre, John M. – 1990
Rank ordering examinees is an easier task for judges than is awarding numerical ratings. A measurement model for rankings based on Rasch's objectivity axioms provides linear, sample-independent and judge-independent measures. Estimates of examinee measures are obtained from the data set of rankings, along with standard errors and fit statistics.…
Descriptors: Comparative Analysis, Error of Measurement, Essay Tests, Evaluators
Peer reviewedWoehr, David J.; And Others – Educational and Psychological Measurement, 1991
Methods for setting cutoff scores based on criterion performance, normative comparison, and absolute judgment were compared for scores on a multiple-choice psychology examination for 121 undergraduates and 251 undergraduates as a comparison group. All methods fell within the standard error of measurement. Implications of differences for decision…
Descriptors: Comparative Analysis, Concurrent Validity, Content Validity, Cutting Scores
