For agencies that deliver high-stakes Second Language (L2) proficiency exams, a research agenda has been undertaken for years to examine the role of rater, task, and rubric as sources of variability into their performance assessments (Lee, 2006; Sawaki & Sinharay, 2013; Xi, 2007; Xi & Mollaun, 2006). However, these challenges are more…

Previous research in second language writing has shown that when scoring performance assessments even trained raters can exhibit significant differences in severity. When raters disagree, using discussion to try to reach a consensus is one popular form of score resolution, particularly in contexts with limited resources, as it does not require…

Pre-writing stage has been considered very significant as it enhances writing performance in that learners can have discussions with the teacher and/or their peers. They can also outline and organize their ideas individually to prepare themselves for writing. This study investigated the effects of using pictures and interactions in the pre-writing…

Raters are central to writing performance assessment, and rater development--training, experience, and expertise--involves a temporal dimension. However, few studies have examined new and experienced raters' rating performance longitudinally over multiple time points. This study uses operational data from the writing section of the MELAB (n =…

The purpose of this study was to investigate ESL students' performance and strategies when writing direct and translated essays. The study also aimed at exploring students' strategies when writing in L2 (English) and L1 (Arabic). The study used a mixture of quantitative and qualitative procedures for data collection and analysis. Adapted strategy…

Research on rater effects in language performance assessments has provided ample evidence for a considerable degree of variability among raters. Building on this research, I advance the hypothesis that experienced raters fall into types or classes that are clearly distinguishable from one another with respect to the importance they attach to…

The main purpose of this research is 1) to establish a framework for the test development and the constructs of writing performance test, 2) to implement a developed writing performance assessment, and 3) to examine the degree of reliability and validity of the assessment tasks and rating scales. Construct-based processing approach to testing…

Alderson (2005) suggests that diagnostic tests should identify strengths and weaknesses in learners' use of language and focus on specific elements rather than global abilities. However, rating scales used in performance assessment have been repeatedly criticized for being imprecise and therefore often resulting in holistic marking by raters…

This investigation examined the impact of opportunity to learn content and skills targeted by a writing assessment on the achievement of English learners (ELs), including the potential for differential impact of increased exposure to literary analysis and writing instruction. Results revealed several factors contributing to students' writing…

Performance assessments have become the norm for evaluating language learners' writing abilities in international examinations of English proficiency. Two aspects of these assessments are usually systematically varied: test takers respond to different prompts, and their responses are read by different raters. This raises the possibility of undue…