This study aims to explore to what extent syntactic complexity predicts holistic scores generated by China's two prominent automated writing evaluation systems--"pigai" and "iWrite," the results of which will have some implications for the validity of these two systems. In the meanwhile, this study targets how syntactic…

For Chinese as a second language (L2 Chinese), there has been little research into "distinguishing features" (Fulcher, 1996; Iwashita et al., 2008) used in scoring L2 Chinese speaking performance. The study reported here investigates the relationship between the distinguishing features of L2 Chinese spoken performances and the scores…

This paper considers the relationship between architecture and construction management students' overall academic abilities (as measured by Weighted Average Marks [WAMs]), their peer ratings for contributions to team design assignments (as measured by an online Self-and-Peer-Assessment [SAPA] tool), and their specific abilities as building…

Various factors contribute to variability in English as a second language (ESL) essay scores and rating processes. Most previous research, however, has focused on score variability in relation to task, rater, and essay characteristics. A few studies have examined variability in essay rating processes. The current study used think-aloud protocols…

The purpose of the 2011 NAEP (National Assessment of Educational Progress) Writing Framework is to describe how the new NAEP Writing Assessment is designed to measure students' writing at grades 4, 8, and 12. As the ongoing national indicator of the academic achievement of students in the United States, NAEP regularly collects information on…

This study examined the relationship of holistic scoring with handwriting legibility, spelling accuracy and number of T-units within compositions written by children in grades 3 through 6 using path analysis. A sample of 223 compositions was rated for handwriting legibility and composition quality, and coded for number of T-units and percentage of…

This article describes a holistic, discourse-based method for assessing the critical thinking skills of undergraduate senior-level finance majors. Rejecting a psychometric assessment approach in which component features of critical thinking are disaggregated, this study is based on a holistic scoring of student memos. Students were asked to…

Offers guidelines and illustrations for developmental writing teachers to help them implement holistic assessment of student writing. Presents a rationale for separating assessment and placement. Discusses the design of writing tasks and the training of raters. Includes a sample scoring rubric. (DMM)

Holistic scoring of essays, a writing assessment method in which trained raters make a single overall judgement of the quality of a piece of writing, has acceptable reliability, face validity, and speed of administration. (Author/CL)

Three different scoring systems are described; these systems are widely used for evaluating students' writing samples. A new model for scoring is presented by comparing and contrasting that model with the other systems. A pilot study of the new model for writing-sample items, scoring keys, and scoring guidelines is discussed. The pilot study was…

The effect of different types of rating task instructions on rater behavior was examined using experts, as opposed to novices, as raters. The experts were instructed to (1) form a global categorical judgment (early hypothesis generation); (2) assess 19 detailed elements; or (3) both. Subjects were 8 dental faculty members who ranged in age from 28…

Scoring systems for direct writing assessment are described. In holistic scoring, a global quality judgment of the writing sample is made. Primary trait scoring, developed by the National Assessment of Educational Progress, is conducted in accordance with specific goals. Analytic scoring identifies characteristics and quality of writing. These…

Holistic scoring enables the evaluation of writing skills based on general impressions of content and style. An experiment in an accounting class shows how it can be applied successfully with a high degree of reliability. (SK)

The question of what variables affect success in debate has long been an area of interest and concern in the forensic community. For many years, it was thought that traditional performance variables--delivery, reasoning, organization, analysis, refutation and use of evidence--were the key factors influencing evaluations of debaters. Some…

This study examined the reliability of holistic scoring for a sample of essays written during the Spring 1985 MCAT administration. Analysis of variance techniques was used to estimate the reliability of scoring and to partition score variance into that due to level differences between papers and to context-specific factors. (Author/LMO)