Using generalizability (G-) theory, this study examined the accuracy and validity of the writing scores assigned to secondary school ESL students in the provincial English examinations in Canada. The major research question that guided this study was: Are there any differences between the accuracy and construct validity of the analytic scores…

The present study was conducted with a twofold purpose. First, I aim to apply the socio-cognitive framework by Shaw and Weir (2007) in order to validate a summative writing test used in a Malaysian ESL secondary school context. Secondly, by applying the framework I also aim to illustrate practical ways in which teachers can gather validity…

Testing the hypotheses that reflective timed-essay prompts should elicit memories of meaningful experiences in students' undergraduate education, and that computer-mediated classroom experiences should be salient among those memories, a combination of quantitative and qualitative research methods paints a richer, more complex picture than either…

Suggests that assessment must be built into the curriculum and focused upon the kinds of skills students need. Considers much educational testing in writing to be reductionist, unrealistic, and detrimental to learning. Critiques writing assessment's trust and reliance on a single or small sample of student work collected and scored outside of a…

Discusses the tension between testing and literacy education, the deleterious influences of testing on the curriculum, and the lack of a firm relationship between improved test scores and improved educational experiences for students. Presents two alternative systems for evaluation of writing skills at the local and state level. Demonstrates the…

Specialists in the field of large-scale, high-stakes writing assessment have, over the last forty years alternately discussed the issue of maximizing either reliability or validity in test design. Factors complicating the debate--such as Messick's (1989) expanded definition of validity, and the ethical implications of testing--are explored. An…

This paper reports on a test that is being used to fulfill a university writing examination requirement for non-native speakers of English. The test, which requires students to read two passages, write short-answer comprehension and synthesis questions, and write an argument essay on a topic related to the passages, replaces a test that was based…

Accuracy in the scoring of writing is critical if standardized tasks are to be used in a national assessment scheme. Three approaches to establishing accuracy (i.e., consensus, consistency, and measurement) exist and commonly large-scale assessment programs of primary school writing demonstrate adjacent agreement consensus rates of between 80% and…