The paper attempts to give an overview of evaluation in higher education institutions with particular emphasis on the faculties of humanities, education and social sciences disciplines at the University of Eswatini (Swaziland) in Southern Africa. It describes the general methodology of evaluation and identifies obstacles and relevant strategies…

Measuring lexical diversity in texts that have different lengths is problematic because length has a significant effect on the number of types a text contains, thus hampering any comparison. Treffers-Daller et al. (2018) recommended a simple solution, namely counting the number of types in a section of a given length that was extracted from the…

The proliferation of large language models represents a paradigm shift in the landscape of automated essay scoring (AES) systems, fundamentally elevating their accuracy and efficacy. This study presents an extensive examination of large language models, with a particular emphasis on the transformative influence of transformer-based models, such as…

For the last century, the quality of large-scale assessment in the United States has been undermined by narrow educational theory and hindered by limitations in technology. As a result, poor assessment practices have encouraged low-level instructional practices that disparately affect students from the most disadvantaged communities and schools.…

Automated Essay Scoring (AES) has garnered a great deal of attention from the rhetoric and composition/writing studies community since the Educational Testing Service began using e-rater[R] and the "Criterion"[R] Online Writing Evaluation Service as products in scoring writing tests, and most of the responses have been negative. While the…

Background: It would be easy to think the technological shifts in the digital revolution are simple incremental progressions in societal advancement. However, the nature of digital technology is resulting in qualitative differences in nearly all parts of daily life. Purpose: This paper investigates how the new possibilities for understanding,…

In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for…

Essays are an important measure of complex learning, but pronouns can confound an author's intended meaning for both readers and text analysis software. This descriptive investigation considers the effect of pronouns on a computer-based text analysis approach, "ALA-Reader," which uses students' essays as the data source for deriving individual and…

We implemented two different approaches of using peer review to support undergraduate essay assignments for students taking large second-year courses in life sciences and biology: a web-based online peer review (OPR) approach and a more traditional face-to-face peer review (FPR) approach that was conducted in tutorial settings. The essays…

The purpose of this theoretical essay is to encourage critical reflection of the relationship of specific reading events to traditional conceptualizations of reader stance. While conventional models of reader stance are useful for considering many aspects of reading, there are reading events that engender stances that appear to lie outside of the…

High-stakes tests take time away from valuable learning activities, narrow the focus of instruction, and imply that science involves memorizing details rather than understanding the natural world. Current tests lead precollege instructors to postpone science inquiry activities until after the last standardized test is completed--often during the…

Automatic Essay Assessor (AEA) is a system that utilizes information retrieval techniques such as Latent Semantic Analysis (LSA), Probabilistic Latent Semantic Analysis (PLSA), and Latent Dirichlet Allocation (LDA) for automatic essay grading. The system uses learning materials and relatively few teacher-graded essays for calibrating the scoring…

A generic approach in automated essay scoring produces scores that have the same meaning across all prompts, existing or new, of a writing assessment. This is accomplished by using a single set of linguistic indicators (or features), a consistent way of combining and weighting these features into essay scores, and a focus on features that are not…

This article details the implementation and use of a "Review Stage" within the CAP (computerised assessment by peers) tool as part of the assessment process for a post-graduate module in e-learning. It reports upon the effect of providing the students with a "second chance" in marking and commenting their peers' essays having been able to view the…

Composition teachers have generally embraced collaborative learning in the years since Kenneth Bruffee published his first article promoting its advantages in "College English" in 1972. But assigning collaboratively written papers in an introductory college writing course is still rare. This study assesses the benefits and drawbacks of assigning a…