Purpose: This study builds a new system for automatically assessing learners' speech elicited from an oral discourse completion task (DCT), and evaluates the prediction capability of the system with a view to better understanding factors deemed influential in predicting speaking proficiency scores and the pedagogical implications of the system.…

VocQGen is an automated tool designed to generate multiple-choice cloze (MCC) questions for vocabulary assessment in second language learning contexts. It leverages several natural language processing (NLP) tools and OpenAI's GPT-4 model to produce MCC items quickly from user-specified word lists. To evaluate its effectiveness, we used the first…

Readers should construct a coherent discourse during reading comprehension. The ability to build coherence has been examined using coherence and cohesion judgment tasks. Although eye-tracking studies have been conducted on building coherence or processing cohesion among native language users, few such studies have been conducted with second…

This study aimed to compare item characteristics and response time between stimulus conditions in computer-delivered listening tests. Listening materials had three variants: regular videos, frame-by-frame videos, and only audios without visuals. Participants were 228 Japanese high school students who were requested to complete one of nine…

The last three decades have seen significant development in understanding and describing the effects of task complexity on learner internal processes. However, researchers have primarily employed behavioral methods to investigate task-generated cognitive load. Being the first to adopt neuroimaging to study second language (L2) task effects, we…

As a result of the Japanese Ministry of Education's recent edict that students' written and spoken English should be assessed in university entrance exams, there is an urgent need for tools to help teachers and students prepare for these exams. Although some commercial tools already exist, they are generally expensive and inflexible. To address…

This study investigated the constructs underlying second language (L2) word association (WA) with regard to three dimensions of lexical competence--size, organization, and accessibility--and the lexical performance of speech. One-hundred and thirteen Japanese learners of English completed a computer-delivered oral WA task along with three…

Email remains a key mode of communication between faculty and students in higher education institutions. Composing appropriate email texts is an important skill for learners; however, little technological support is available for the pragmatic aspect of email communication -- the ways in which social context influences language choices.…

Many tertiary language courses have some online components as part of their blended learning initiatives. These online activities could be part of assessments, and online quizzes/tests can be used as preparation for the demanding final written examination. This study attempts to shed some light on the effects of online tests on final examination…

Despite an emphasis on oral communication in most foreign language classrooms, the resource-intensive nature (i.e. time and manpower) of speaking tests hinder regular oral assessments. A possible solution is the development of a (semi-) automated scoring system. When it is used in conjunction with human raters, the consistency of computers can…

Introduction: This paper describes the exploratory case and initial evaluation of the computer-based testing (CBT) prototype. The advantage of CBT over paper-based testing (PBT) is that it allows us to control the order of questions and provides test takers with continuous tasks capturing their thought processes. Additionally, their response…

This article investigates the relative effectiveness of online and paper-based formative assessment in the learning of English as a second language. For this study, 74 Japanese university second language (L2) students took online quizzes, while 71 were given paper-based quizzes to complete. The quiz content was drawn from learning material found…

Background: Research on the test-taker perception of assessments has been conducted under the assumption that negative test-taker perception may influence test performance by decreasing test-taking motivation. This assumption, however, has not been verified in the field of language testing. Building on expectancy-value theory, this study explored…

Phonological processing of written characters has been recognized as a crucial element in acquiring literacy in any language, both native and foreign. This study aimed to assess Japanese primary school students' phoneme-grapheme recognition skills using both paper-based and touch-interface tests. Differences between the two test formats and the…

This study investigated through an online survey how sign language practitioners changed their sign language assessment practices during the COVID-19 pandemic. The survey consisted of five sections and 29 questions overall. It was provided in written English and German as well as in International Sign and was administered online between October…