This paper studies the use of Reinforcement Learning (RL) policies for optimizing the sequencing of online learning materials to students. Our approach provides an end to end pipeline for automatically deriving and evaluating robust representations of students' interactions and policies for content sequencing in online educational settings. We…

The main purpose of this quantitative-qualitative content analysis study was to compare IELTS and TOEFL listening and reading tests based on the representation of the learning objectives of Revised Bloom's taxonomy. To this end, 12 Academic IELTS listening and reading tests and 12 TOEFL iBT listening and reading tests were analyzed qualitatively…

The purpose of this mixed-methods study was to explore Iranian IELTS candidates' strengths and weaknesses in IELTS Speaking Test in terms of IELTS's four speaking assessment criteria, namely Fluency and Coherence (FlC), Lexical Resource (LR), Grammar Range and Accuracy (GRA), and Pronunciation (Pro). It also aimed to examine the discourse features…

Common estimators of interrater reliability include Pearson product-moment correlation coefficients, Spearman rank-order correlations, and the generalizability coefficient. The purpose of this study was to examine the accuracy of estimators of interrater reliability when varying the true reliability, number of scale categories, and number of…

Item Response Theory (IRT) is a ubiquitous model for understanding humans based on their responses to questions, used in fields as diverse as education, medicine and psychology. Large modern datasets offer opportunities to capture more nuances in human behavior, potentially improving test scoring and better informing public policy. Yet larger…

A large body studies into individual differences in second language learning has shown that success in second language learning is strongly affected by a set of relevant learner characteristics ranging from the age of onset to motivation, aptitude, and personality. Most studies have concentrated on a limited number of learner characteristics and…

The maze task is a psycholinguistic experimental procedure that measures real-time incremental sentence processing. The task has recently been tested as a language learning tool with promising results. Therefore, the present study examines the merits of a contextualized version of this task: the story maze. The findings are consistent with…

Previous research in cognitive diagnostic assessment (CDA) of L2 reading ability has been frequently conducted using large-scale English proficiency exams (e.g., TOEFL, MELAB). Using CDA, it is possible to analyze individual learners' strengths and weaknesses in multiple attributes (i.e., knowledge, skill, strategy) measured at the item level.…

The study utilised a fine-grained diagnostic checklist to assess first-year undergraduates in Hong Kong and evaluated its validity and usefulness for diagnosing academic writing in English. Ten English language instructors marked 472 academic essays with the checklist. They also agreed on a Q-matrix, which specified the relationships among the…

Within a Dynamic System Theory (DST) approach, it is assumed that language is in a constant flux, but that differences in the degree of variability can give insight into the developmental process. This longitudinal case study focuses on intra-individual variability in accuracy rates and complexity measures in Finnish learner language. The study…

This paper reports a set of Monte Carlo simulations designed to evaluate the main claims made by Laufer and Nation about the Lexical Frequency Profile (LFP). Laufer and Nation claim that the LFP is a sensitive and reliable tool for assessing productive vocabulary in L2 speakers, and they suggest it might have a serious role to play in diagnostic…

Probabilistic models with more than one latent variable are designed to report profiles of skills or cognitive attributes. Testing programs want to offer additional information beyond what a single test score can provide using these skill profiles. Many recent approaches to skill profile models are limited to dichotomous data and have made use of…

This study assessed the factor structure of the LanguEdge™ test and the invariance of its factors across language groups. Confirmatory factor analyses of individual tasks and subsets of items in the four sections of the test, Listening, Reading, Speaking, and Writing, was carried out for Arabic-, Chinese-, and Spanish-speaking test takers. Two…