Evaluating the performance of Large Language Models (LLMs) is a critical yet challenging task, particularly when aiming to avoid subjective assessments. This paper proposes a framework for leveraging subjective metrics derived from the class textual materials across different semesters to assess LLM outputs across various tasks. By utilizing…

Research on the effectiveness of Intelligent Tutoring Systems (ITSs) suggests that automatic hint generation has the best effect on learning outcomes when hints are provided on the level of intermediate steps. However, ITSs for programming tasks face the challenge to decide on the granularity of steps for feedback, since it is not a priori clear…

This paper undertakes a thorough evaluation of ChatGPT's code generation capabilities, contrasting them with those of human programmers from both educational and software engineering standpoints. The emphasis is placed on elucidating its importance in these intertwined domains. To facilitate a robust analysis, we curated a novel dataset comprising…

Code quality is a critical aspect of programming, as high-quality code is easier to maintain, and code maintenance constitutes the majority of software costs. Consequently, code quality should be emphasized in programming education. While previous research has identified numerous code quality defects commonly made by students, the current state…

The current educational system relies heavily on manual grading, posing challenges such as delayed feedback and grading inaccuracies. Automated grading tools (AGTs) offer solutions but come with limitations. To address this, "GRAD-AI" is introduced, an advanced AGT that combines automation with teacher involvement for precise grading,…

Automated assessment is well-established within computer science courses but largely absent from human--computer interaction courses. Automating the assessment of human--computer interaction (HCI) is challenging because the coursework tends not to be computational but rather highly creative, such as designing and implementing interactive…

We conducted a systematic literature review on automated grading and feedback tools for programming education. We analysed 121 research papers from 2017 to 2021 inclusive and categorised them based on skills assessed, approach, language paradigm, degree of automation, and evaluation techniques. Most papers assess the correctness of assignments in…

"Stochastic programs," which are programs that produce probabilistic output, are a pivotal paradigm in various areas of CS education from introductory programming to machine learning and data science. Despite their importance, the problem of automatically grading such programs remains surprisingly unexplored. In this paper, we formalize…

This dissertation focuses on strategies and techniques to enhance code comprehension skills among students enrolled in introductory computer science courses (CS1 and CS2). We propose a novel tutoring system, "DeepCodeTutor," designed to improve the code comprehension abilities of novices. DeepCodeTutor employs scaffolded self-explanation…

Background and Context: Auto-graders are praised by novice students learning to program, as they provide them with automatic feedback about their problem-solving process. However, some students often make random changes when they have errors in their code, without engaging in deliberate thinking about the cause of the error. Objective: To…

Data transfer between isolated clusters is imperative for cybersecurity education, research, and testing. Such techniques facilitate hands-on cybersecurity learning in isolated clusters, allow cybersecurity students to practice with various hacking tools, and develop professional cybersecurity technical skills. Educators often use these remote…

Programming demands a variety of cognitive skills, and mastering these competencies is essential for success in computer science education. The importance of formative feedback is well acknowledged in programming education, and thus, a diverse range of techniques has been proposed to generate and enhance formative feedback for programming…

When the new programming language Pascal was developed in the 1970's, Walter Gander did not like it because because many features which he appreciated in prior programming languages were missing in Pascal. For example the block structure was gone, there were no dynamical arrays, no functions or procedures were allowed as parameters of a procedure,…

For the last decades, programming courses are being taught in nearly every educational sector. Students are now more likely to use an e-learning platform compared to traditional system because of lower internet costs, remote access, and faster communication facilities. For a programming course studied in both manual and e-learning platforms,…

Algorithm design is a vital skill developed in most undergraduate Computer Science (CS) programs, but few research studies focus on pedagogy related to algorithms coursework. To understand the work that has been done in the area, we present a systematic survey and literature review of CS Education studies. We search for research that is both…