This scope-review presents the milestones of how Hierarchical Rater Models (HRMs) become operable to used in automated essay scoring (AES) to improve instructional evaluation. Although essay evaluations--a useful instrument for evaluating higher-order cognitive abilities--have always depended on human raters, concerns regarding rater bias,…

Objectively scoring constructed-response items on educational assessments has long been a challenge due to the use of human raters. Even well-trained raters using a rubric can inaccurately assess essays. Unfolding models measure rater's scoring accuracy by capturing the discrepancy between criterion and operational ratings by placing essays on an…

Evaluating the quality of university exam papers is crucial for universities seeking institutional and program accreditation. Currently, exam papers are assessed manually, a process that can be tedious, lengthy, and in some cases, inconsistent. This is often due to the focus on assessing only the formal specifications of exam papers. This study…

The goal of this paper is to find better ways to estimate the internal consistency reliability of scores on tests with a specific type of design that are often encountered in practice: tests with constructed-response items clustered into sections that are not parallel or tau-equivalent, and one of the sections has only one item. To estimate the…

As the COVID-19 pandemic motivated a shift to virtual teaching, exams have increasingly moved online too. Detecting cheating through collusion is not easy when tech-savvy students take online exams at home and on their own devices. Such online at-home exams may tempt students to collude and share materials and answers. However, online exams'…

Aim/Purpose: This article investigates the process of identifying and correcting hallucinations in ChatGPT-4's recall of student-written responses as well as its evaluation of these responses, and provision of feedback. Effective prompting is examined to enhance the pre-evaluation, evaluation, and post-evaluation stages. Background: Advanced Large…

In large lecture courses, it can be challenging to imagine assessing student learning in ways other than multiple-choice exams and traditional point-based grading. Inspired by major pedagogical principles shared by Maria Montessori and Thomas Jefferson and supported by current understandings of effective teaching, assessment was reimagined in a…

Intelligent systems of essay grading constitute important tools for educational technologies. They can significantly replace the manual scoring efforts and provide instructional feedback as well. These systems typically include two main parts: a feature extractor and an automatic grading model. The latter is generally based on computational and…

Efficient constructed response (CR) scoring requires both accuracy and speed from human raters. This study was designed to determine if setting scoring rate expectations would encourage raters to score at a faster pace, and if so, if there would be differential effects on scoring accuracy for raters who score at different rates. Three rater groups…

Higher-order thinking skills (HOTS) is one of the skills needed for 21st century challenges, especially for students. The aim of this study was to describe HOTS in students, especially the cognitive domain of analyzing and evaluating. This is a descriptive quantitative study employing a Pretest-Posttest One Group research design. The experiment…

The educational value of portfolios as assessments has been widely acknowledged across the higher education sector and literature as providing a platform to promote student-centred and reflective learning (Brown, 1997; Snadden & Thomas, 1998; Karlowicz, 2000). While there is plentiful research investigating the benefits of providing portfolios…

Despite the increasing ubiquity of computer-based tests, many general qualifications examinations remain in a paper-based mode. Insufficient and unequal digital provision across schools is often identified as a major barrier to a full adoption of computer-based exams for general qualifications. One way to overcome this barrier is a gradual…

The Ethical Reasoning in Action program leveraged program assessment data to further promote student learning based on their effective educational framework for ethical reasoning: The Eight Key Question (8KQ) strategy. A cross-disciplinary team of five core researchers launched the first constructivist qualitative inquiry into students' ethical…

Education for Sustainable Development (ESD) serves as a key accelerator for achieving the Sustainable Development Goals (SDGs), emphasizing systems thinking as an essential competency that must be cultivated in the learning process. This study investigates students' systems thinking skills within the ESD framework through assessments on…

The paper attempts to give an overview of evaluation in higher education institutions with particular emphasis on the faculties of humanities, education and social sciences disciplines at the University of Eswatini (Swaziland) in Southern Africa. It describes the general methodology of evaluation and identifies obstacles and relevant strategies…