E-rater[R] is an automated essay scoring system that uses natural language processing techniques to extract features from essays and to model statistically human holistic ratings. Educational Testing Service has investigated the use of e-rater, in conjunction with human ratings, to score one of the two writing tasks on the TOEFL-iBT[R] writing…

With the United States' adoption of a standards-based approach to education, most attention has focused on the large-scale, high-stakes assessments intended to measure students' mastery of standards for accountability purposes. Less attention has been paid to the role of standards-based assessments in the classroom. The purpose of this paper is to…

The "Versant Spanish[TM] Test" applies state-of-the-art technology to language assessment in a quick and efficient tool designed for the purpose of "assessing an individual's ability to speak and understand spoken Spanish". According to the developers, the Pearson Education Group, the test is equally useful in academic as well as professional…

Noting the widespread use of multiple-choice items in tests in English language education in Thailand, this study compared their effect against that of constructed-response items. One hundred and fifty-two university undergraduates took a test of English structure first in constructed-response format, and later in three, stem-equivalent…

Despite the increasing interest in and need for test information for use in instructional practice and student learning, there have been few attempts to systematically link a diagnostic approach to English for academic purposes (EAP) writing instruction and assessment. In response to this need for research, this study examined the extent to which…

Motivated by cognitive theories of graph comprehension, this study systematically manipulated characteristics of a line graph description task in a speaking test in ways to mitigate the influence of graph familiarity, a potential source of construct-irrelevant variance. It extends Xi (2005), which found that the differences in holistic scores on…

The competencies required for conducting assessment in the educational context have recently been reformulated in view of social constructivist perspectives and the acknowledgement of the role of classroom assessment in promoting learning. These changes have impacted the knowledge base language assessors need to obtain, and hence the contents of…

Information provided by examination of the skills that underlie holistic scores can be used not only as supporting evidence for the validity of inferences associated with performance tests but also as a way to improve the scoring rubrics, descriptors, and benchmarks associated with scoring scales. As fluency is considered a critical, perhaps…

This paper presents evidence that supports the valid use of scores from fully automatic tests of spoken language ability to indicate a person's effectiveness in spoken communication. The paper reviews the constructs, scoring, and the concurrent validity evidence of "facility-in-L2" tests, a family of automated spoken language tests in Spanish,…

Advances in natural language processing (NLP) and automatic speech recognition and processing technologies offer new opportunities for language testing. Despite their potential uses on a range of language test item types, relatively little work has been done in this area, and it is therefore not well understood by test developers, researchers or…

The study reported here examined the evaluation criteria used to assess the proficiency and effectiveness of the language produced in an oral performance test of English conducted in an American university context. Empirical methods were used to analyze qualitatively and quantitatively transcriptions of the Oral English Tests (OET) of 44…

This investigation intends to assess the effectiveness of a lexical recognition test (Meara & Buxton, 1987) as a placement tool that distinguishes among levels of two groups of students: Spanish heritage language learners (HLL) and second language learners (SLL). Three hundred and thirty university students from four different levels completed a…

A meta-analysis was conducted on the effects of multiple-choice and open-ended formats on L1 reading, L2 reading, and L2 listening test performance. Fifty-six data sources located in an extensive search of the literature were the basis for the estimates of the mean effect sizes of test format effects. The results using the mixed effects model of…

This article reviews Assessing Comprehension and Communication in English State-to-State for English Language Learners ("ACCESS for ELLs"[R]), which is a large-scale, high-stakes, standards-based, and criterion-referenced English language proficiency test administered in the USA annually to more than 840,000 English Language Learners (ELLs), in…

Generalizability of writing scores has always been a longstanding concern in L2 writing assessment. A number of studies have been conducted to investigate this topic during the last two decades. However, with the introduction of new test methods, such as reading-to-write tasks, generalizability studies need to focus on the score accuracy of…