ERIC Number: EJ1467995
Record Type: Journal
Publication Date: 2025
Pages: 10
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-1545-679X
Available Date: 0000-00-00
AI-Powered Learning Support: A Study of Retrieval-Augmented Generation (RAG) Chatbot Effectiveness in an Online Course
Guido Lang; Tan Gürpinar
Information Systems Education Journal, v23 n2 p4-13 2025
This study investigates the effectiveness of a Retrieval-Augmented Generation (RAG) chatbot to enhance learning and engagement in a self-paced, asynchronous online R programming course. To contextualize the development and potential of RAG chatbots, we conducted a literature review on existing approaches and their use in educational settings. Following this, a chatbot powered by generative artificial intelligence (GenAI) was designed to provide tailored conceptual explanations and code examples based on course materials, addressing a range of student inquiries. To evaluate its effectiveness, the study analyzed chatbot interaction logs and survey responses collected at the end of the course. Results showed that students with greater prior knowledge of the subject matter were more likely to engage with the chatbot, primarily seeking help on advanced topics not covered in the course lectures. Overall, students expressed high satisfaction with the chatbot, particularly valuing its ability to provide helpful explanations that are based on the course materials. This study highlights the potential of GenAI, and RAG chatbots specifically, to enhance online education and provides practical insights for future implementations.
Descriptors: Artificial Intelligence, Technology Uses in Education, Program Effectiveness, Online Courses, Student Satisfaction, Programming
Information Systems and Computing Academic Professionals. Box 488, Wrightsville Beach, NC 28480. e-mail: publisher@isedj.org; Web site: http://isedj.org
Publication Type: Journal Articles; Reports - Research; Tests/Questionnaires
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A