This study evaluates the ability of large language models (LLMs) to deliver criterion-based grading and examines the impact of prompt engineering with detailed criteria on grading. Using well-established human benchmarks and quantitative analyses, we found that even free LLMs achieve criterion-based grading with a detailed understanding of the…

Evaluation is an essential component of the learning process when discerning learning situations. Assessing natural language responses, like short answers, takes time and effort. Artificial intelligence and natural language processing advancements have led to more studies on automatically grading short answers. In this review, we systematically…

Automatic grading of short answers is an important task in computer-assisted assessment (CAA). Recently, embeddings, as semantic-rich textual representations, have been increasingly used to represent short answers and predict the grade. Despite the recent trend of applying embeddings in automatic short answer grading (ASAG), there are no…

Intelligent systems of essay grading constitute important tools for educational technologies. They can significantly replace the manual scoring efforts and provide instructional feedback as well. These systems typically include two main parts: a feature extractor and an automatic grading model. The latter is generally based on computational and…

This review compares generative artificial intelligence with five representative educational technologies in history and concludes that AI technology can become a knowledge producer and thus can be utilized as educative AI to enhance teaching and learning outcomes. From a historical perspective, each technological breakthrough has affected…

Educational technology innovations leveraging large language models (LLMs) have shown the potential to automate the laborious process of generating and analysing textual content. While various innovations have been developed to automate a range of educational tasks (eg, question generation, feedback provision, and essay grading), there are…

We explore how different components of an Automatic Short Answer Grading (ASAG) model affect the model's ability to generalize to questions outside of those used for training. For supervised automatic grading models, human ratings are primarily used as ground truth labels. Producing such ratings can be resource heavy, as subject matter experts…

Purpose: The use of generative artificial intelligence (genAi) language models such as ChatGPT to write assignment text is well established. This paper aims to assess to what extent genAi can be used to obtain guidance on how to avoid detection when commissioning and submitting contract-written assignments and how workable the offered solutions…

In postsecondary education institutions, where innovative technologies continually reshape research and pedagogical approaches, the integration of generative artificial intelligence (GenAI) tools presents promising avenues for enhancing student learning experiences. This study assesses the efficacy of integrating GenAI tools, specifically…

This study conducts a comprehensive quantitative evaluation of OpenAI's language model, ChatGPT 4, for grading Task 2 writing of the IELTS exam. The objective is to assess the alignment between ChatGPT's grading and that of official human raters. The analysis encompassed a multifaceted approach, including a comparison of means and reliability…

The purpose of this study is to examine the impact of artificial intelligence (AI) on online assessment in the context of opportunities and threats based on the literature. To this end, 19 articles related to the AI tool ChatGPT and online assessment were analysed through rapid literature review. In the content analysis, the themes of "AI's…

The development, use, and timely promotion of Open Education (OE) has been effective in addressing myriad educational concerns, including inclusivity, accessibility and learning achievement, among many others. However, limited information exists in the literature concerning how OE could enhance Generative Artificial Intelligence (GenAI), which is…

Conversational Intelligent Tutoring Systems (ITSs) are expensive to develop. While simple online courseware could be easily authored by teachers, the authoring of conversational ITSs usually involves a team of experts with different expertise, including domain experts, linguists, instruction designers, programmers, artists, computer scientists,…

Human-guided machine learning can improve computing intelligence, and it can accurately assist humans in various tasks. In education research, artificial intelligence (AI) is applicable in many situations, such as predicting students' learning paths and strategies. In this study, we explore the benefits of repetitive practice of short-answer…

Massive open online courses (MOOCs) provide hundreds of students with teaching materials, assessment tools, and collaborative instruments. The assessment activity, in particular, is demanding in terms of both time and effort; thus, the use of artificial intelligence can be useful to address and reduce the time and effort required. This paper…