ERIC Number: EJ1460858
Record Type: Journal
Publication Date: 2025-Feb
Pages: 16
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1059-0145
EISSN: EISSN-1573-1839
Available Date: 2024-11-13
Applying Generative Artificial Intelligence to Critiquing Science Assessments
Ha Nguyen1; Jake Hayward2
Journal of Science Education and Technology, v34 n1 p199-214 2025
High-quality science assessments are multi-dimensional. They promote disciplinary practices, core ideas, cross-cutting concepts, and science sense-making. In this paper, we investigate the feasibility of using generative artificial intelligence (GenAI), specifically multimodal large language models (MLLMs), to annotate and provide improvement ideas for K-12 science assessments. The AI-generated annotations critique how the assessments align with the three dimensions of the Next Generation Science Standards (NGSS) and suggest ideas to elicit students' science sense-making. We outline our process with various prompting strategies: few-shot and zero-shot learning (prompting with and without examples), chain of thought (eliciting the MLLM's reasoning), and sampling strategies (outputting high or low level of randomness). Overall, the AI annotations can reason about the alignment between the assessments and NGSS dimensions and overlap with annotations from K-12 educators. Annotations generated with few-shot learning generally score higher overall and provide more details than zero-shot prompts. Further, interviews with science teachers reveal that the MLLM annotations can support teachers' reflection on instructional practices and assessment revision. We discuss the application of MLLMs to develop three-dimensional science assessments.
Descriptors: Science Tests, Criticism, Artificial Intelligence, Technology Uses in Education, Computer Software, Science Education, Standards, Kindergarten, Elementary Secondary Education, Computational Linguistics, Prompting, Alignment (Education), Science Teachers, Teaching Methods, Teacher Attitudes, Test Construction, Educational Improvement
Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Publication Type: Journal Articles; Reports - Research
Education Level: Early Childhood Education; Elementary Education; Kindergarten; Primary Education; Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF)
Authoring Institution: N/A
Grant or Contract Numbers: 2422286
Author Affiliations: 1University of North Carolina at Chapel Hill, School of Education, Chapel Hill, USA; 2Utah State University, Department of Instructional Technology & Learning Sciences, Logan, USA