Artificial Intelligence (AI) based assessments are commonly used in a variety of settings including business, healthcare, policing, manufacturing, and education. In education, AI-based assessments undergird intelligent tutoring systems as well as many tools used to evaluate students and, in turn, guide learning and instruction. This chapter…

This research report provides a description of the processes of evaluating the "deployability" of automated scoring (AS) systems from the perspective of large-scale educational assessments in operational settings. It discusses a comprehensive psychometric evaluation that entails analyses that take into consideration the specific purpose…

The use of new digital formats in language learning and testing improves both the learning and acquisition skills development process of language tests. The process of listening comprehension is considered one of the most complex in the field of Computer Aided Learning Language (CALL) because it relates to multimodal learning channels and brain…

Automating the production of questions for assessment and self-assessment has become recently an active field of study. The use of Semantic Web technologies has certain advantages over other methods for question generation and thus is one of the most important lines of research for this problem. The aim of this paper is to provide an overview of…

We present a tool for the creation and curation of C-tests. C-tests are an established tool in language proficiency testing and language learning. They require examinees to complete a text in which the second half of every second word is replaced by a gap. We support teachers and test designers in creating such tests through a web-based system…

Most research in psycholinguistics relies on online measures such as reading time to inform and test theories of language comprehension. However, the value of offline measures such as question-answering performance is sometimes overlooked in sentence processing work. Consequently, psycholinguists do not yet understand how the tasks and measures…

E-rater[R] is an automated essay scoring system that uses natural language processing techniques to extract features from essays and to model statistically human holistic ratings. Educational Testing Service has investigated the use of e-rater, in conjunction with human ratings, to score one of the two writing tasks on the TOEFL-iBT[R] writing…

Advances in natural language processing (NLP) and automatic speech recognition and processing technologies offer new opportunities for language testing. Despite their potential uses on a range of language test item types, relatively little work has been done in this area, and it is therefore not well understood by test developers, researchers or…

The aim of this article is to present an approach for generating tests in an automatic way. Although other methods have been already reported in the literature, the proposed approach is based on ontologies, representing both domain and multimedia knowledge. The article also reports on a prototype implementation of this approach, which…

The authors report an implemented environment for computer-assisted authoring of test items and provide a brief discussion about the applications of NLP techniques for computer assisted language learning. Test items can serve as a tool for language learners to examine their competence in the target language. The authors apply techniques for…

Feedback on assessment tasks has an important part to play in underpinning student learning. Online assessment enables instantaneous feedback to be given so that the student can act on it immediately. However, concern has been expressed that e-assessment tasks (especially multiple-choice questions) can encourage surface-learning. Several projects…

This paper reports the results of a dual-task experiment which investigates the nature of working memory resources used in sentence comprehension. Participants read sentences of varying syntactic complexity (containing subject-and object-extracted relative clauses) while remembering one or three nouns (similar to or dissimilar from the…

The aim of this study was to find ways to improve online reading and vocabulary learning. Various new types of online reading comprehension and vocabulary development programs and tests were compared in this study to consider how useful they may be for guiding individual or classroom L2 vocabulary instruction. It explored how these programs seek…

Adaptation techniques can be applied not only to the multimedia contents or navigational possibilities of a course, but also to the assessment. In order to facilitate the authoring of adaptive free-text assessment and its integration within adaptive web-based courses, Adaptive Hypermedia techniques and Free-text Computer Assisted Assessment are…