The literature on Automated Essay Scoring (AES) systems has provided useful validation frameworks for any assessment that includes AES scoring. Furthermore, evidence for the scoring fidelity of AES systems is accumulating. Yet questions remain when appraising the scoring performance of AES systems. These questions include: (a) which essays are…

Writing essays and receiving feedback can be useful for fostering students' learning and motivation. When faced with large class sizes, it is desirable to identify students who might particularly benefit from feedback. In this article, we tested the potential of Latent Semantic Analysis (LSA) for identifying poor essays. A total of 14 teaching…

This study is designed to gather information concerning a possible relationship between how dental students prefer to take in and communicate new information and how they prefer to be assessed. Though there are numerous references in the literature regarding the learning styles of students there are also references to the inaccuracy of such…

This study used think-aloud protocols to compare essay-rating processes across holistic and analytic rating scales in the context of China's College English Test Band 6 (CET-6). A group of 9 experienced CET-6 raters scored the same batch of 10 CET-6 essays produced in an operational CET-6 administration twice, using both the CET-6 holistic…

To determine if quizzes containing higher order thinking questions are related to critical thinking and test performance when utilised in conjunction with an immersion approach to instruction and effort-based grading, sections of an "Educational Psychology" course were assigned to one of three quizzing conditions. Quizzes contained…

This study has attempted to investigate the administered written tests in the language department of Islamic Azad University of Hamedan, Iran from validity, practicality and reliability points of view. To this end two steps were taken. First, examining 112 tests, we knew that the face validity of 50 tests had been threatened, 9 tests lacked…

Scoring models for the "e-rater"® system were built and evaluated for the "TOEFL"® exam's independent and integrated writing prompts. Prompt-specific and generic scoring models were built, and evaluation statistics, such as weighted kappas, Pearson correlations, standardized differences in mean scores, and correlations with…

This article details the implementation and use of a "Review Stage" within the CAP (computerised assessment by peers) tool as part of the assessment process for a post-graduate module in e-learning. It reports upon the effect of providing the students with a "second chance" in marking and commenting their peers' essays having been able to view the…

This investigation focused on a group of 17 faculty drawn from disciplines in the humanities social sciences and natural sciences. Using in-depth interviews, focus group discussions and qualitative coding strategies, this study examined how faculty conceptualized the term critical thinking (CT), and how they assessed for it in general education…

The purpose of the current study was to analyze the relationship between automated essay scoring (AES) and human scoring in order to determine the validity and usefulness of AES for large-scale placement tests. Specifically, a correlational research design was used to examine the correlations between AES performance and human raters' performance.…

Multiple choice questions (MCQs) are becoming more common in UK psychology departments and the need to assess their reliability is apparent. Having examined the reliability of MCQs in our department we faced many questions from colleagues about why we were examining reliability, what it was that we were doing, and what should be reported when…

Discusses the basic concepts of testing and item development and the application of alternative assessments to information literacy content for library instruction. Topics include reliability; validity; statistical analysis; selected response, including checklists, rank order, or simple match; constructed response; essays; and complex assessments.…

In this study, we investigated a variety of Reader effects that may influence the validity of ratings assigned to AP® English Literature and Composition essays. Specifically, we investigated whether Readers exhibit changes in their levels of severity and accuracy, and their use of individual scale categories over time. We refer to changes in these…

The purpose of this study was to determine if there were differences in postsecondary marketing student performance on essay tests based on test format (i.e., computer-based or handwritten). Specifically, the variables of performance, test completion time, and gender were explored for differences based on essay test format. Results of the study…

This study examined 412 college students' essay performance on two prompt types, a traditional prose essay and a type incorporating graphics, modeled on those from English Language Challenge Examination (ELCE) developed for the University of Southern California (USC). The majority of the participants were international students at USC. Each…