The plagiarism detection programs SafeAssign and Turnitin are commonly used at the collegiate level to detect improper use of outside sources. In order to determine whether either program is superior, this study evaluated the programs using four standards: (1) the ability to detect legitimate plagiarism, (2) the ability to avoid false positives,…

In this paper, I describe the design and evaluation of automated essay scoring (AES) models for an institution's writing placement program. Information was gathered on admitted student writing performance at a science and technology research university in the northeastern United States. Under timed conditions, first-year students (N = 879) were…

This study investigated the use of automated essay scoring (AES) to identify at-risk students enrolled in a first-year university writing course. An application of AES, the "Criterion"[R] Online Writing Evaluation Service was evaluated through a methodology focusing on construct modelling, response processes, disaggregation, extrapolation,…

This paper introduces "CTutor", an automated writing evaluation (AWE) tool for detecting breakdowns in local coherence and reports on a study that applies it to the writing of Chinese L2 English learners. The program is based on Centering theory (CT), a theory of local coherence and salience. The principles of CT are first introduced and…

This study investigates the potential of a software tool based on Latent Semantic Analysis (LSA; Landauer, McNamara, Dennis, & Kintsch, 2007) to automatically evaluate complex German texts. A sample of N = 94 German university students provided written answers to questions that involved a high amount of analytical reasoning and evaluation.…

This article considers the claim that machine scoring of writing test responses agrees with human readers as much as humans agree with other humans. These claims about the reliability of machine scoring of writing are usually based on specific and constrained writing tasks, and there is reason for asking whether machine scoring of writing requires…

The hurried pace of the modern classroom does not permit formative feedback on writing assignments at the frequency or quality recommended by the research literature. One solution for increasing individual feedback to students is to incorporate some form of computer-generated assessment. This study explores the use of automated assessment of…

Online marking and feedback systems are critical for providing timely and accurate feedback to students and maintaining the integrity of results in large class teaching. Previous investigations have involved much in-house development and more consideration is needed for deploying or customising off the shelf solutions. Furthermore, keeping up to…

This paper presents an empirical evaluation of automated writing evaluation (AWE) feedback used for L2 academic writing teaching and learning. It introduces the Intelligent Academic Discourse Evaluator (IADE), a new web-based AWE program that analyzes the introduction section to research articles and generates immediate, individualized, and…

Describes a study in which 12 students' papers were critiqued by both WordPerfect's "Grammatik" grammar checking software and by real English/business communication professors. Finds significant limitations to the effectiveness of the software as a guide for student writers. (SR)

Examines the use of Prep-Editor, a computer program to help undergraduate science writing students communicate with their peers about drafts. Finds that the program did not increase time spent on various writing activities. Notes that the Prep group reported a number of computer-related problems, whereas the non-Prep group reported more…

This report provides a two-part evaluation of the IntelliMetric[SM] automated essay scoring system based on its performance scoring essays from the Analytic Writing Assessment of the Graduate Management Admission Test[TM] (GMAT[TM]). The IntelliMetric system performance is first compared to that of individual human raters, a Bayesian system…

E-rater[R] has been used by the Educational Testing Service for automated essay scoring since 1999. This paper describes a new version of e-rater (V.2) that is different from other automated essay scoring systems in several important respects. The main innovations of e-rater V.2 are a small, intuitive, and meaningful set of features used for…

Reports on the satisfaction of college news writing students with a computer program which combines reporting simulation with a writing analysis of the students' news stories. Finds that students rated the computer program significantly ahead of the paper version of the exercise. Suggests providing more detailed feedback in the computer program.…

A case is made against the use of computer-assisted text analysis, including style analyzers and grammar checkers, with nonproficient second-language or basic-skills student writers. The inaccuracy of analysis, limited feedback, and type of revision behavior encouraged are cited. (33 references) (Author/LB)