Language proficiency diagnosis is essential to extract fine-grained information about the linguistic knowledge states and skill mastery levels of test takers based on their performance on language tests. Different from comprehensive standardized tests, many language learning apps often revolve around word-level questions. Therefore, knowledge…

The needs for precisely estimating a student's academic performance have been emphasized with an increasing amount of attention paid to Intelligent Tutoring System (ITS). However, since labels for academic performance, such as test scores, are collected from outside of ITS, obtaining the labels is costly, leading to label-scarcity problem which…

Advances in machine learning (ML) are poised to contribute to our understanding of the linguistic processes associated with successful reading comprehension, which is a critical aspect of children's educational success. We used ML techniques to investigate and compare associations between children's reading comprehension and 260 linguistic…

This article presents a systematic instructional design procedure to develop a Computer-Assisted Vocabulary Acquisition (CAVA) software in a consistent and reliable way. The design process involves ADDIE fundamental elements: Analyse, Design, Development, Implementation, and Evaluation. This article will describe in detail each of the five ADDIE…

A computer program was developed to estimate speed-accuracy response models for dichotomous items. This report describes how the models are estimated and how to specify data and input files. An example using data from a listening section of an international language test is described to illustrate the modeling approach and features of the computer…

This study investigated the effects of auditory and audiovisual text presentation on the three levels of mental representations assumed in theories of discourse processing. A sample of 106 children aged 7, 9, and 11 years listened to 16 short narrative texts, 8 of which were accompanied by a series of pictures illustrating the content.…

Diagnostic classification models (aka cognitive or skills diagnosis models) have shown great promise for evaluating mastery on a multidimensional profile of skills as assessed through examinee responses, but continued development and application of these models has been hindered by a lack of readily available software. In this article we…

One core area of research in Second Language Acquisition is the identification and definition of developmental stages in different L2s. For L2 French, Bartning and Schlyter (2004) presented a model of six morphosyntactic stages of development in the shape of grammatical profiles. The model formed the basis for the computer program Direkt Profil…

This review paper constitutes the first step within a larger research effort to develop an interactive pragmatics learning tool for second and foreign language (L2) learners and users of English. The tool will primarily endeavor to support pragmatics learning within the language use domain "workplace." Given this superordinate objective,…

In this research, we investigated the suitability of implementing "e-rater"® automated essay scoring in a high-stakes large-scale English language testing program. We examined the effectiveness of generic scoring and 2 variants of prompt-based scoring approaches. Effectiveness was evaluated on a number of dimensions, including agreement…

This study aimed to (1) integrate relevant language learning models and theories, (2) construct a theoretical model of college students' English learning performance, and (3) assess the model fit between empirically observed data and the theoretical model proposed by the researchers of this study. Subjects of this study were 1,129 Taiwanese EFL…

Describes a three-phase process model for interface design, drawing on practices developed in the software industry and adapting them for computer-based languages tests. Describes good practice in initial design, emphasizes the importance of usability testing, and argues that only through following a principled approach to interface design can the…

By order of the Civil Service Commission of Belgium, Didascalia, the University of Antwerp developed an adaptive system for evaluating second-language proficiency. A computer model based on probability and information theory was developed; the first module of the ATLAS test currently assesses approximately 15,000 civil servants yearly. (one…

The ability of the Rasch model computer program BIGSTEPS to perform concurrent calibration and differences in the outcomes between vertical equating using a common item anchoring method and a common item concurrent calibration method was examined. The data for the investigation came from two Chinese language tests that researchers wished to…

Reviews current and developing technology uses that are relevant to language assessment and discusses examples of recent linguistic applications from the laboratory at the Educational Testing Service. The processes of language test development are described and the functions they serve from the perspective of a large testing organization are…