Teamwork skills are commonly evaluated by human assessors, which can be logistically challenging and resource intensive. Technological advancements provide an opportunity for a new assessment method -- virtual behavioural simulations with self-scoring algorithms. This study explores whether a rule-based algorithm can match human assessors at…

The interpretations of test scores in secure, high-stakes environments are dependent on several assumptions, one of which is that examinee responses to items are independent and no enemy items are included on the same forms. This paper documents the development and implementation of a C#-based application that uses Natural Language Processing…

This article introduces Xiao-Shih, the first intelligent question answering bot on Chinese-based massive open online courses (MOOCs). Question answering is critical for solving individual problems. However, instructors on MOOCs must respond to many questions, and learners must wait a long time for answers. To address this issue, Xiao-Shih…

This study fit a Naïve Bayesian classifier to the words of exam items to predict the Bloom's taxonomy level of the items. We addressed five research questions, showing that reasonably good prediction of Bloom's level was possible, but accuracy varies across levels. In our study, performance for Level 2 was poor (Level 2 items were misclassified…

Building on previous studies investigating the multidimensional nature of lexical use in task-based L2 performance, this study clarified the roles that the distinct lexical features play in predicting vocabulary proficiency in a corpus of L2 Oral Proficiency Interviews (OPI). A total of 85 OPI samples were rated by three separate raters based on a…

Educational content labeled with proper knowledge components (KCs) are particularly useful to teachers or content organizers. However, manually labeling educational content is labor intensive and error-prone. To address this challenge, prior research proposed machine learning based solutions to auto-label educational content with limited success.…

Game-based learning environments are designed to provide effective and engaging learning experiences for students. Predictive student models use trace data extracted from students' in-game learning behaviors to unobtrusively generate early assessments of student knowledge and skills, equipping game-based learning environments with the capacity to…

Although text readability has traditionally been measured based on simple linguistic features, recent studies have employed natural language processing techniques to develop new readability formulas that better represent theoretical accounts of reading processes. This study evaluated the construct validity of different readability formulas,…

Dialogue act (DA) classification plays an important role in understanding, interpreting and modeling dialogue. Dialogue acts (DAs) represent the intended meaning of an utterance, which is associated with the illocutionary force (or the speaker's intention), such as greetings, questions, requests, statements, and agreements. In natural language…

While hierarchical machine learning approaches have been used to classify texts into different content areas, this approach has, to our knowledge, not been used in the automated assessment of text difficulty. This study compared the accuracy of four classification machine learning approaches (flat, one-vs-one, one-vs-all, and hierarchical) using…

Open-ended comprehension questions are a common type of assessment used to evaluate how well students understand one of multiple documents. Our aim is to use natural language processing (NLP) to infer the level and type of inferencing within readers' answers to comprehension questions using linguistic and semantic features within their responses.…

This article presents the first-ever big data study of the student evaluation of teaching (SET) using artificial intelligence (AI). We train natural language processing (NLP) models on 1.6 million student evaluations from the US and the UK. We address two research questions: (1) are these models able to predict student ratings from the student…

This study examines the extent to which instructions to self-explain vs. "other"-explain a text lead readers to produce different forms of explanations. Natural language processing was used to examine the content and characteristics of the explanations produced as a function of instruction condition. Undergraduate students (n = 146)…

Influenced by COVID-19, online learning has become one of the most important forms of education in the world. In the era of intelligent education, knowledge tracing (KT) can provide excellent technical support for individualized teaching. For online learning, we come up with a new knowledge tracing method that integrates mathematical exercise…

An NLP algorithm was developed to assess question quality to inform feedback on questions generated by students within iSTART (an intelligent tutoring system that teaches reading strategies). A corpus of 4575 questions was coded using a four-level taxonomy. NLP indices were calculated for each question and machine learning was used to predict…