Objectively scoring constructed-response items on educational assessments has long been a challenge due to the use of human raters. Even well-trained raters using a rubric can inaccurately assess essays. Unfolding models measure rater's scoring accuracy by capturing the discrepancy between criterion and operational ratings by placing essays on an…

The goal of this study is to model pauses extracted from writing keystroke logs as a way of characterizing the processes students use in essay composition. Low-level timing data were modeled, the interkey interval and its subtype, the intraword duration, thought to reflect processes associated with keyboarding skills and composition fluency.…

Adverbs require a great deal of effort to be mastered, and even the most advanced users of that language have difficulty in using them correctly (Narita & Sugiura, 2006; Peacock, 2010; Lei, 2012; Leedham & Cai, 2013). The purpose of this study is to find out to what extent relatively high proficiency level EFL learners use different types…

Automated scoring systems are typically evaluated by comparing the performance of a single automated rater item-by-item to human raters. This presents a challenge when the performance of multiple raters needs to be compared across multiple items. Rankings could depend on specifics of the ranking procedure; observed differences could be due to…

A testing-based learning strategy is one that relies on the act of recalling (i.e., testing) information after exposure, and interleaving is a strategy in which the learning materials are presented in a serial order (e.g., "texts 1, 2, 3, 1, 2, 3, 1, 2, 3") versus a blocked order (e.g., "texts 1, 1, 1, 2, 2, 2, 3, 3, 3").…

The improvisations needed to adapt to COVID-19 teaching and learning conditions affected students and faculty alike. This study uses chaos theory and improvisation to examine an undergraduate communication research methods course that was initially delivered synchronously/face-to-face and then transitioned to asynchronous/online in March 2020.…

The literature on Automated Essay Scoring (AES) systems has provided useful validation frameworks for any assessment that includes AES scoring. Furthermore, evidence for the scoring fidelity of AES systems is accumulating. Yet questions remain when appraising the scoring performance of AES systems. These questions include: (a) which essays are…

In the current study, two pools of 250 essays, all written as a response to the same prompt, were rated by two groups of raters (14 or 15 raters per group), thereby providing an approximation to the essay's true score. An automated essay scoring (AES) system was trained on the datasets and then scored the essays using a cross-validation scheme. By…

Numerous studies have shown that simply being in another culture does not guarantee the development of intercultural competence. Students need guidance to seek out opportunities to engage and to make sense of those experiences. Reflection has become a popular methodology to assist students with this. Unfortunately, students often do not know how…

Over two semesters short essay prompts were developed for use with the Graphical Interface for Knowledge Structure (GIKS), an automated essay scoring system. Participants were students in an undergraduate-level online introductory statistics course. The GIKS compares students' writing samples with an expert's to produce keyword occurrence and…

Since researchers investigated automatic scoring systems in writing assessments, they have dealt with relationships between human and machine scoring, and then have suggested evaluation criteria for inter-rater agreement. The main purpose of my study is to investigate the magnitudes of and relationships among indices for inter-rater agreement used…

The goal of this study was to assess the effectiveness of 2 different types of intervention aimed at improving written argumentative synthesis by integrating conflicting information from different sources. Both interventions combined the use of a guide with collaborative practice, but one of them also included explicit strategy instruction. Only…

Social cognitive learning theory has shown that observational learning positively influences essay writing development in high-school students, and that self-efficacy impacts on motivation. This study investigated the relative contribution of model observation, model evaluation, post-submission feedback, and factors relating to self-efficacy, as…

Literature revealed that the patterns/methods of scoring essay tests had been criticized for not being reliable and this unreliability is more likely to be more in internal examinations than in the external examinations. The purpose of this study is to find out the effects of analytical and holistic scoring patterns on scorer reliability in…

Objectives: To investigate the relation between the students' scores in MCQs and MEQs of the summative assessment in pediatrics at the College of medicine KKU. Introduction: Student assessment is the most difficult task in medicine since it is ultimately related to human life and safety. Assessment can take different types of formats with…