This controlled experimental study investigated the interaction of variables associated with rating the pronunciation component of high-stakes English-language-speaking tests such as IELTS and TOEFL iBT. One hundred experienced raters who were all either familiar or unfamiliar with Brazilian-accented English or Papua New Guinean Tok Pisin-accented…

Listening comprehension in English, as one of the most fundamental skills, has an essential role in the process of learning English. Mokken scale analysis (MSA) is a probabilistic-nonparametric approach to item response theory (IRT) which determines the one-dimensionality and scalability of test. Mokken scaling techniques are a useful tool for…

Synthetically generated speech (SGS) has become an integral part of our oral communication in a wide variety of contexts. It can be generated instantly at a low cost and allows precise control over multiple aspects of output, all of which can be highly appealing to second language (L2) assessment developers who have traditionally relied upon human…

Communities of practice is a term that has begun gaining grounds in various settings especially in business and recently in education (Argyris, 2008; Göhlich, 2016; Lave & Wegner, 1991). The purpose of this case study is to look into how students measure their personal perceptions of their educational community; how they rate their educator…

This paper investigates the attitudes of 246 English teachers from Outer and Expanding Circles (EC and OC) towards the global spread of English and the challenges underpinning international proficiency tests. Using a questionnaire, supplemented with interviews with a smaller group of teachers, this study explores how EC and OC teachers perceive…

Syntactic complexity is an important measure of second language (L2) writing proficiency (Larsen--Freeman, 1978; Lu, 2011). Large-grained indices such as the mean length of T-unit (MLTU) have been used with the most consistency in L2 writing studies (Ortega, 2003). Recently, indices such as MLTU have been criticized, both for the difficulty in…

Threats to construct validity should be reduced to a minimum. If true, sources of bias, namely raters, items, tests as well as gender, age, race, language background, culture, and socio-economic status need to be spotted and removed. This study investigates raters' experience, language background, and the choice of essay prompt as potential…

This study aims to examine the relationship between reading comprehension and lexical and grammatical knowledge among English as a foreign language students by using an Artificial Neural Network (ANN). There were 825 test takers administered both a second-language reading test and a set of psychometrically validated grammar and vocabulary tests.…

Developing a diagnostic tool within the diagnostic measurement framework is the optimal approach to obtain multidimensional and classification-based feedback on examinees. However, end users may seek to obtain diagnostic feedback from existing item responses to assessments that have been designed under either the classical test theory or item…

English proficiency is valued in Japan for its facilitation of participation in the international community. However, average proficiency remains low despite strategic efforts of Japan's Ministry of Education. Based on research indicating that culture may impact learning, this quantitative study investigated the impact of culture on English…

The aim of this study was to develop, for the benefit of both test takers and test score users, enhanced "TOEFL ITP"® test score reports that go beyond the simple numerical scores that are currently reported. To do so, we applied traditional scale anchoring (proficiency scaling) to item difficulty data in order to develop performance…

Stakeholders of language tests are often interested in subscores. However, reporting a subscore is not always justified; a subscore should provide reliable and distinct information to be worth reporting. When a subscore is used for decisions across multiple levels (e.g., individual test takers and schools), it needs to be justified for its…

Although several studies have explored the effects of task sequencing on second language (L2) production, there is no established set of criteria to sequence tasks for learners in L2 writing classrooms. This study examined the effect of simple ?complex task sequencing manipulated along both resource-directing (± number of elements) and…

This study had two aims. The first was to explain the process of using the Rasch measurement model to validate tests in an easy-to-understand way for those unfamiliar with the Rasch measurement model. The second was to validate two final exams with several shared items. The exams were given to two groups of students with slightly differing English…

Multiple test tasks are available for assessing L2 collocation knowledge. However, few studies have investigated the characteristics of a variety of recognition and recall tasks of collocation simultaneously, and most research on L2 collocations has focused on verb-noun and adjective-noun collocations. This study investigates (1) the relative…