The use of concurrent data elicitation procedures (e.g. think-alouds, eye-tracking, response time) to investigate learners' cognitive processing and processes is becoming more prominent in research designs as researchers seek to acquire a better understanding of how second language (L2) learners process L2 data (e.g. Martin et al., 2019; Rogers,…

This paper investigates what matters to medical domain experts when setting standards on a language for specific purposes (LSP) English proficiency test: the Occupational English Test's (OET) writing sub-test. The study explores what standard-setting participants value when making performance judgements about test candidates' writing responses,…

This study aims to investigate English as a foreign language (EFL) learners' use of context information under various noise conditions. The participants were seven Japanese undergraduate students. A noise test adopted from the Speech-Perception-in-Noise (SPIN) test (Kalikow et al., 1977) was used. Four signal-to-noise ratio (SNR) conditions (SNR =…

The paper reports on trends in language testing taking place over the years and aim at critical perspectives of testing and promoting inclusion, equity and justice. It begins with critical theories by Messick, Foucault and Bourdieu, leading to critical language testing (CLT) which focused on consequences and uses of tests. Given the power of tests…

The present experiment was composed of both qualitative and quantitative methods and aimed at investigating Reading Metacognitive Strategy Awareness (RMSA) and use by the aid of three inventory techniques including a questionnaire, a think-aloud protocol, and a Computerized RMSA System. The computerized system was specifically designed and…

Although an accurate measure of vocabulary size is integral to understanding the proficiency of language learners, the validity of multiple-choice (M/C) vocabulary tests to determine this has been questioned due to users guessing correct answers which inflates scores. In this paper the nature of guessing and partial knowledge used when taking the…

In language performance tests, raters are important as their scoring decisions determine which aspects of performance the scores represent; however, raters are considered as one of the potential sources contributing to unwanted variability in scores (Davis, 2012). Although a great number of studies have been conducted to unpack how rater…

The present study reports reading strategies of Arabic L1 EFL college-level learners at three reading ability levels: high, medium, and low. Thirty students were asked to take the reading section of the TOFEL to determine their reading abilities. Then two participants from each level were selected randomly to go through the think-aloud protocol…

Artificial intelligence (AI) technologies have the potential to reduce the workload for the second language (L2) teachers and test developers. We propose two AI distractor-generating methods for creating Chinese vocabulary items: semantic similarity and visual similarity. Semantic similarity refers to antonyms and synonyms, while visual similarity…

Listen-to-summarize cloze (LSC) tasks are not rare in high-stakes language tests; however, we know little about what test-takers do during the tasks. In our study, we examined 16 students' think-aloud protocols (TAPs) which were recorded while they were completing two LSC tasks. The analysis of the 16 TAPs indicated iterative cognitive processes…

The aim of this study was to present a protocol for the validation of the Language ENvironment Analysis (LENA) System's conversational turn count (CTC) for Vietnamese speakers. Ten families of children aged between 22 and 42 months, recruited near Ho Chi Minh City, participated in this project. Each child wore the LENA audio recorder for a full…

This article examines the test-taking strategies of high- and low-scoring Chinese-speaking participants when they answer English multiple-choice reading comprehension questions. Thirty-two participants took a TOEIC reading test, provided think-aloud protocols, and joined a post-task interview. The data come primarily from qualitative analysis and…

There has been a substantial body of research, in L2, documenting the central importance of strategic knowledge in reading comprehension and exploring the variables likely to influence strategy use by readers. All this research has, however, been conducted within the single-text tradition of reading research. With a movement toward the…

Corpora are well-known for the affordance to make linguistic regularities salient. Since the coinage of the term 'data-driven learning' (DDL) in the 1990s, much has been done to investigate the effects of DDL on learning vocabulary, most notably lexico-grammatical patterns. However, less researched is how learners construct vocabulary knowledge…

Since its inception, pragmatic competence has been studied vastly; the majority of these studies, however, have examined pragmatic production, and pragmatic comprehension is chiefly under-researched. This lack of adequate research also stands true for the underlying pragmatic comprehension processes and strategies. Therefore, the present…