Short-Answer Grading (SAG) is a time-consuming task for teachers that automated SAG models have long promised to make easier. However, there are three challenges for their broad-scale adoption: A technical challenge regarding the need for high-quality models, which is exacerbated for languages with fewer resources than English; a usability…

Automated Essay Scoring (AES) is a well-studied problem in Natural Language Processing applied in education. Solutions vary from handcrafted linguistic features to large Transformer-based models, implying a significant effort in feature extraction and model implementation. We introduce a novel Automated Machine Learning (AutoML) pipeline…

Automated Essay Scoring (AES) is a well-studied problem in Natural Language Processing applied in education. Solutions vary from handcrafted linguistic features to large Transformer-based models, implying a significant effort in feature extraction and model implementation. We introduce a novel Automated Machine Learning (AutoML) pipeline…

Conversational agents have been widely used in education to support student learning. There have been recent attempts to design and use conversational agents to conduct assessments (i.e., conversation-based assessments: CBA). In this study, we developed CBA with constructed and selected-response tests using Rasa--an artificial intelligence-based…

The transition from paper-based tests to corresponding computer-administered tests allows for the incorporation of improved interfaces that support response making. The main research question is whether innovative interfaces affect test response time and/or response accuracy. This study compared performance on banked cloze tests using a…

The purpose of this study is to investigate the quality design criteria for developing a Massive Open Online Course (MOOC). Currently, there are limited studies that highlight the required design criteria for the MOOC programming courses. A descriptive analysis was conducted to examine the characteristics of the three important quality design…

Task solving processes and changes in these processes have long been expected to provide valuable information about children's performance in school. This article used electronic tangibles (concrete materials that can be physically manipulated) and a dynamic testing format (pretest, training, and posttest) to investigate children's task solving…

Developing online test environments for e-learning for mobile devices will be useful to increase drill practice opportunities. In order to provide a drill practice environment for calculus using an online math test system, such as STACK, we develop a flickable math input interface that can be easily used on mobile devices. The number of taps…

There is some evidence that a scrolling design may reduce breakoffs in mobile web surveys compared to a paging design, but there is little empirical evidence to guide the choice of the optimal number of items per page. We investigate the effect of the number of items presented on a page on data quality in two types of questionnaires: with or…

Multistage testing, or MST, was developed as an alternative to computerized adaptive testing (CAT) for applications in which it is preferable to administer a test at the level of item sets (i.e., modules). As with CAT, the simulation technique in MST plays a critical role in the development and maintenance of tests. "MSTGen," a new MST…

A design and application of adaptive online exam system are carried out in this paper. Adaptive exam systems determine different question sets automatically and interactively for each student and measure their competence on a certain area of discipline instead of comparing their gains with each other. Through an adaptive exam technique, a…

The computer-based "Graduate Record Examinations"® ("GRE"®) revised General Test includes interactive item types and testing environment tools (e.g., test navigation, on-screen calculator, and help). How well do test takers understand these innovations? If test takers do not understand the new item types, these innovations may…

Between March and June of 2014, the Smarter Balanced Assessment Consortium conducted a field test of its new online assessment system. Thirteen participating states provided the results of surveys given to students and adults involved in the Field Test. Overall, more than 70% of test coordinators in each of seven states indicated that the Field…

What makes customization so appealing? Is it because the content is tailored or because the user feels greater agency? Study 1 tested these propositions with a news-aggregator Website that was either personalized (system-tailored), customized (user-tailored), or neither. Power users rated content quality higher when it had a customizable…

Up-to-date spelling process was assessed using typical spelling-to-dictation tasks, where children's performance was evaluated mainly in terms of spelling error scores. In the present work a simple graphical computer interface is reported, aiming to investigate the effects of input modality (e.g. visual and verbal) in word spelling. The software…