A variety of cognitive diagnosis approaches (CDAs) have been used by researchers to extract fine-grained information about the linguistic knowledge states and skill mastery levels of test takers based on their performance on language tests. Although some of these approaches have been in use in the psychometric community for quite some time, many…

Many studies have indicated that at least 2 raters should score writing assessments to improve interrater reliability. However, even for assessments that characteristically demonstrate high levels of rater agreement, 2 raters of the same essay can occasionally report different, or discrepant, scores. If a single score, typically referred to as an…