We present work in progress on a multimodal dialog system for English language assessment using a modular cloud-based architecture adhering to open industry standards. Among the modules being developed for the system, multiple modules heavily exploit machine learning techniques, including speech recognition, spoken language proficiency rating,…

There has been an increased interest in the impact of unmotivated test taking on test performance and score validity. This has led to the development of new ways of measuring test-taking effort based on item response time. In particular, Response Time Effort (RTE) has been shown to provide an assessment of effort down to the level of individual…

The clinical assessment of mental disorders can be a time-consuming and error-prone procedure, consisting of a sequence of diagnostic hypothesis formulation and testing aimed at restricting the set of plausible diagnoses for the patient. In this article, we propose a novel computerized system for the adaptive testing of psychological disorders.…

The present study investigates the best factor for controlling the item difficulty of multiple-choice English vocabulary questions generated by an automatic question generation system. Three factors are considered for controlling item difficulty: (1) reading passage difficulty, (2) semantic similarity between the correct answer and distractors,…

Computer-based assessments have considerable potential in assessment. This study explores the perceptions of 500 post-graduate students, noting gender differences where appropriate, in relation to the role of computer applications in formative assessment. Using questionnaires and interviews, the data reveal that the student perceptions of the role…

Exams assessment is one of the most tedious work for university teachers all over the world. Multiple choice theses make exams assessment a little bit easier, but the teacher cannot prepare more than 3-4 variants; in this case, the possibility of students for cheating from one another becomes a risk for "objective assessment outcome." On…

Proponents of reframing argue that prophetic pragmatism entails redirecting contemporary education reforms. While this judgment may defend our professional standing, it overlooks the consequences of redirecting reform's appeals to global competition, which preclude public participation in defining the goals and measures of literacy education. This…

Polytomous items are typically described by multiple category-related parameters; situations, however, arise in which a single index is needed to describe an item's location along a latent trait continuum. Situations in which a single index would be needed include item selection in computerized adaptive testing or test assembly. Therefore single…

Open-ended tasks are common in Science, Technology, Engineering and Mathematics (STEM) education. However, as far as we know, no tools have been developed to assist in the assessment of the solution process of open-ended questions. In this paper, we propose the use of analysis of traces as a tool to address this need. To illustrate this approach,…

This study analyzed design and technological issues encountered by online course instructors as they taught 120 online courses while working with a designated instructional designer. After developing the courses and their approval in compliance with a set, standardized rubric, the implementation of the 120 courses brought out issues that required…

Causal sequences can be segmented into cause and effect. However, some argue causal relations in discourse are by default in "effect-cause" order. Others claim "cause-effect" order is easier to process and the default way of expressing causality, due to iconicity. We conducted experiments testing participants' production…

The divisibility of the reading has led many researchers to examine whether reading construct is a unitary skill that cannot be divided into separate sub-skills, or it is composed of underlying components. In these studies, however, participants have generally been located in a product-oriented testing environment where their actual reading…

Historically, the implementation of research-based assessments (RBAs) has been a driver of educational change within physics and helped motivate adoption of interactive engagement pedagogies. Until recently, RBAs were given to students exclusively on paper and in class; however, this approach has important drawbacks including decentralized data…

Using game-based learning strategies, such as quiz-style PowerPoint games as an e-learning and teaching pedagogy can make a positive impact on how students learn, how they process and retain information, and how they interact with digital media. However, little is known about their impact on students' information literacy development. This…

This paper presents an innovative method to tackle the automatic evaluation of programming assignments with an approach based on well-founded assessment theories (Classical Test Theory (CTT) and Item Response Theory (IRT)) instead of heuristic assessment as in other systems. CTT and/or IRT are used to grade the results of different items of…