This study evaluated the efficacy of two proposed methods in an operational standard-setting study conducted for a high-stakes language proficiency test of the U.S. government. The goal was to seek low-cost modifications to the existing Yes/No Angoff method to increase the validity and reliability of the recommended cut scores using a convergent…

The importance of functional adequacy as an essential component of L2 proficiency has been observed by several authors (Pallotti, 2009; De Jong, Steinel, Florijn, Schoonen, & Hulstijn, 2012a, b). The rationale underlying the present study is that the assessment of writing proficiency in L2 is not fully possible without taking into account the…

This study investigates the validity of assessing L2 pragmatics in interaction using mixed methods, focusing on the evaluation inference. Open role-plays that are meaningful and relevant to the stakeholders in an English for Academic Purposes context were developed for classroom assessment. For meaningful score interpretations and accurate…

Partial dictation is a measure of EFL listening proficiency that can be easily constructed, administered, and scored by EFL teachers. However, it is controversial whether this form of test measures lower-order abilities exclusively or involves both lower- and higher-order abilities. In order to answer this question, a study was designed to examine…

Gist identification or coherent situation model construction performance is an important criterion not only in L1 and L2 reading comprehension assessment but also in every aspect of discourse processing. Whereas most previous L2 research has investigated schematic knowledge of readers about relations within a text, more recent studies have used…

English has been an official language of Hong Kong since it was first introduced to the region in the 1840s. Since then, its influence has invariably been strong and remains so even after 1997, when the sovereignty of the region was returned to China. This special background of English in Hong Kong heavily influences the policy and practices of…

This study examined the effects of training on peer assessment and comments provided regarding oral presentations in EFL (English as a Foreign Language) classrooms. In Study 1, both the treatment and control groups received instruction on skill aspects, but only the treatment group was given an additional 40-minute training on how to rate…

Illustrates some of the issues that have arisen in the implementation of outcomes-based systems of assessment and reporting. Two examples from school and adult immigrant education in Australia are provided that demonstrate some of the political and technical problems involved in implementation in both high- and low-stakes contexts. Outlines ways…

Investigated the impact of performance conditions on perceptions of task difficulty in a test of spoken language, in light of the cognitive complexity framework proposed by Skehan (1998). Candidates performed narrative tasks whose characteristics and the conditions under which they were performed were manipulated, and the impact of these on task…

Considers the ecological validity of tests in predicting the effects of speech and language impairment on communicative performance in real-world settings and the need for systematic observation of communicative behaviors, particularly of individuals from other cultures. Discusses whether standardized testing and systematic observation are…

Summarizes research on self-assessment in second-language testing using a meta-analysis on 60 correlations reported in second-language-testing literature. Self-assessments and teacher assessments of recently instructed English-as-a-Second-Language learners' functional English skills revealed differential validities for self-assessment and teacher…