This research investigates the effects of rater experience on performance ratings in language testing using a systematic review of studies published from 1985 to 2017. Based on a comprehensive literature search of 14 databases, we identified sixteen relevant papers. With these we conducted a narrative review to conceptualize a theoretical…

A brief summary of standard setting knowledge is presented, derived from about 20 methods that utilize a judgmental review process, the approach most relevant to the standard-setting strategies proposed in this special issue. Criteria for judging effectiveness and critiques of the methods discussed in the issue are offered. (SLD)

The articles of this special issue propose two methods of deriving an initial standard and one method for determining the extent to which the standard should include compensation. Much work remains to be done on further development of the methods and the larger issues of policy regarding performance assessment. (SLD)

The three standard-setting approaches described in this special issue are summarized and contrasted: (1) judgmental policy capturing; (2) the extended Angoff method; and (3) the dominant profile method. An integrative summary of findings is followed by recommendations for modifying the methods. (SLD)

Although the quest for better measurement of individual job performance has generated considerable empirical research in industrial and organizational psychology, the feeling persists that a good job is not really being done in measuring job performance. This research project investigated the effects of differences in both individual and systems…

Anecdotal and conceptual research on assessor training programs in Australia since 1990 was reviewed to determine research priorities for the next 3 years and to identify the research's implications for policymakers and practitioners concerned with training assessors for three settings: formal training, workplace training, and human resource…