ERIC Number: ED674600
Record Type: Non-Journal
Publication Date: 2025-Jul
Pages: 8
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: 0000-00-00
One Model to Score Them All: Unified Scoring of Learning Strategies with LLMs
Andreea Dutulescu1,2; Stefan Ruseti1,2; Mihai Dascalu1,2; Danielle S. McNamara3
Grantee Submission, Paper presented at the International Conference of Educational Data Mining (18th, Palermo, Italy, Jul 20-23, 2025)
The assessment of student responses to learning-strategy prompts, such as self-explanation, summarization, and paraphrasing, is essential for evaluating cognitive engagement and comprehension. However, manual scoring is resource-intensive, limiting its scalability in educational settings. This study investigates the use of Large Language Models for automating the evaluation of student responses based on expert-defined rubrics. We fine-tune open-source LLMs on annotated datasets to predict expert ratings across multiple scoring rubrics, ensuring consistency and efficiency in assessment. Our findings indicate that multi-task fine-tuning, which involves training a single model across multiple scoring tasks, consistently outperforms single-task training by enhancing generalization and mitigating overfitting. This advantage is particularly noticeable in recent architectures, where multi-task training enables robust performance across diverse evaluation criteria. Notably, our Llama 3.2 3B model achieved high performance, outperforming a 20x larger zero-shot model while maintaining feasibility for deployment on consumer-grade hardware, emphasizing the potential for scalable AI-driven assessment solutions. This research contributes to open education by fine-tuning open-source models and publicly releasing trained models, training scripts, and evaluation frameworks. The proposed approach supports automated, reproducible, and scalable assessment of learning strategies, facilitating timely feedback for students and reducing the burden on educators. [This paper was published in: "Proceedings of the 18th International Conference on Educational Data Mining, Palermo, Italy, July, 2025," edited by Caitlin Mills et al., International Educational Data Mining Society, 2025, pp. 496-502. Additional funding by the project "Romanian Hub for Artificial Intelligence -- HRIA", Smart Growth, Digitization and Financial Instruments Program, 2021-2027, MySMIS no. 334906, and the grant of the Academy of Romanian Scientists, AOSR-TEAMS-IV Edition 2025-2026 "Digital Transformation in Science."]
Related Records: ED675648
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305T240035
Data File: URL: https://github.com/upb-nlp/EDM-LLM-Scoring
Department of Education Funded: Yes

Peer reviewed
Direct link
