It is common practice for assessment programs to organize qualifying sessions during which the raters (often known as "markers" or "judges") demonstrate their consistency before operational rating commences. Because of the high-stakes nature of many rating activities, the research community tends to continuously explore new…

Background: Research studies (e.g., Phakiti, Language Testing 25(2):237-272, 2008a; Phakiti, Language Assessment Quarterly 5(1): 20-42, 2008b; Purpura, Learner strategy use and performance on language tests: a structural equation modeling approach, 1999) conducted on the construct validation of Bachman and Palmer's (1996) strategic competencehas…

Willingness to communicate (WTC) has recently been researched as a dynamic variable, with some investigations viewing it as a complex dynamic system (CDS). One important property associated with CDS is the notion of attractor states, which are characterized by stable patterns of behavior. The present study employed an idiodynamic method to monitor…

The aim of this study is to construct a model of a simple lexical network showing the strength and asymmetry of the connections between vocabulary items in the L2 mental lexicon of Japanese learners. The study focuses on eight nouns and investigates how they are networked, and whether the existing network structure formed by these nouns would be…

There is limited research on teaching narrative writing skills to students with moderate to severe intellectual disability. In the current study, we used a multiple probe across participants single case design to evaluate the effects of an intervention package comprised of modeling, story templates, and self-graphing, on the inclusion of story…

The purpose of this study was to determine the extent to which performance on the TOEFL iBT speaking section is associated with other indicators of Japanese university students' abilities to communicate orally in an academic English environment and to determine which components of oral ability for these tasks are best assessed by TOEFL iBT. To…

Newer statistical procedures are typically introduced to help address the limitations of those already in practice or to deal with emerging research needs. Quantile regression (QR) is introduced in this paper as a relatively new methodology, which is intended to overcome some of the limitations of least squares mean regression (LMR). QR is more…

The study compares EFL writers' processes in composing reading-based writing (RW) and graph-based writing (GW) tasks developed for a university English proficiency exam. Think-aloud protocols and interviews of ten university-level nonnative English-speaking writers were collected to explore writers' composing processes. The results revealed that…

This study examined the relationship between scores on the TOEFL Internet-Based Test (TOEFL iBT[R]) and academic performance in higher education, defined here in terms of grade point average (GPA). The academic records for 2594 undergraduate and graduate students were collected from 10 universities in the United States. The data consisted of…

This paper illustrates a method of item analysis used to identify discriminating multiple-choice items in placement data. The data come from two rounds of pilots given to both SHL students and Spanish as a Second Language (SSL) students. In the first round, 104 items were administered to 507 students. After discarding poor items, the second round…

With the increasing use of integrated tasks in assessing writing, more and more research studies have been conducted to examine the construct validity of such tasks. Previous studies have largely focused on reading-writing tasks, while relatively little is known about graph-writing tasks. This study examines second language (L2) writers'…

The "TOEFL Junior"® Standard Test is an assessment that measures the degree to which middle school-aged students learning English as a second language have attained proficiency in the academic and social English skills representative of English-medium instructional environments. The assessment measures skills in three areas: listening…

Motivated by cognitive theories of graph comprehension, this study systematically manipulated characteristics of a line graph description task in a speaking test in ways to mitigate the influence of graph familiarity, a potential source of construct-irrelevant variance. It extends Xi (2005), which found that the differences in holistic scores on…

Results of some of the Modern Language Association (MLA) Cooperative Foreign Language Tests, administered to students of Spanish, German, and French in grades 9-12 in the Pennsbury School District of Pennsylvania are compared with the national norms. Although most of the comparison is set up in tablees a summary of the analysis is included.…

Multifaceted Rasch analysis was used to determine whether bias was evident in the way a group of raters graded two different versions of an oral interaction test, undertaken by the same candidates. Results indicate that certain raters consistently rated the tape version of the test more harshly while others rated the live one more harshly. (10…