Studies on automatic scoring systems in writing assessments have also evaluated the relationship between human and machine scores for the reliability of automated essay scoring systems. This study investigated the magnitudes of indices for inter-rater agreement and discrepancy, especially regarding human and machine scoring, in writing assessment.…

This literature review attempts to survey representative studies within the context of L2 speaking assessment that have contributed to the conceptualization of rater cognition. Two types of studies are looked at: 1) studies that examine "how" raters differ (and sometimes agree) in their cognitive processes and rating behaviors, in terms…

Elicited imitation (EI) has been widely used to examine second language (L2) proficiency and development and was an especially popular method in the 1970s and early 1980s. However, as the field embraced more communicative approaches to both instruction and assessment, the use of EI diminished, and the construct-related validity of EI scores as a…

While researchers generally quantify the amount of information that learners recall correctly in order to measure reading comprehension, the unit of analysis adopted to score the recall protocol differs. Whether and how different scoring systems bring about a different picture of L2 reading comprehension, however, remains unexplored. This study…

The 2003 NAAL is a complex assessment with several components and various types of data. The primary purpose of this publication is to describe the assessment's key features and data types. Thus, the publication covers the critical concepts and features carried over from the 1992 assessment, as well as those new to the 2003 assessment--for…

Using criterion-referenced test item data collected in an empirical study, differences in item selection between Cox and Vargas' pretest-posttest validity index and a latent trait approach (evaluation of the item information function for the mastery score) are analyzed. (Author/GK)