In the last two decades, computational thinking has gained wide relevance in international educational systems. The inclusion of this new type of thinking poses educational challenges with some underlying research questions that need to be answered to meet these challenges with quality. Thus, this study focuses on analyzing the difficulties that…

Over the past decade, social scientists have adapted computational methods for qualitative text analysis, with the hope that they can match the accuracy and reliability of hand coding. The emergence of GPT and open-source generative large language models (LLMs) has transformed this process by shifting from programming to engaging with models using…

Background: Analyzing student short-answer written justifications to conceptually challenging questions has proven helpful to understand student thinking and improve conceptual understanding. However, qualitative analyses are limited by the burden of analyzing large amounts of text. Purpose: We apply dense and sparse Large Language Models (LLMs)…

ChatGPT is a new technological tool with the potential to impact education. Using Vergnaud's notion of "use schemes," we analyzed three interviews with engineering students who discovered ChatGPT and started to develop initial utilization schemes of the tool. Results showed that there were three domains of use of ChatGPT: (a) in…

Aim/Purpose: This paper is part of a multi-case study that aims to test whether generative AI makes an effective coding assistant. Particularly, this work evaluates the ability of two AI chatbots (ChatGPT and Bing Chat) to generate concise computer code, considers ethical issues related to generative AI, and offers suggestions for how to improve…

Background: High-quality feedback is crucial for academic success, driving student motivation and engagement while research explores effective delivery and student interactions. Advances in artificial intelligence (AI), particularly natural language processing (NLP), offer innovative methods for analyzing complex qualitative data such as feedback…

Physics education researchers are interested in using the tools of machine learning and natural language processing to make quantitative claims from natural language and text data, such as open-ended responses to survey questions. The aspiration is that this form of machine coding may be more efficient and consistent than human coding, allowing…

Effective collaboration and teamwork skills are critical in high-risk sectors, as deficiencies in these areas can result in injuries and risk of death. To foster the growth of these vital skills, immersive learning spaces have been created to simulate real-world scenarios, enabling students to safely improve their teamwork abilities. In such…

This study investigates the potential of automated classification using prompt-based learning approaches with transformer models (large language models trained in an unsupervised manner) for a domain-specific classification task. Prompt-based learning with zero or few shots has the potential to (1) make use of artificial intelligence without…

Large Language Models (LLMs), such as ChatGPT, have become widely popular for various software engineering tasks, including programming, testing, code review, and program comprehension. However, their impact on improving software quality in educational settings remains uncertain. This article explores our experience teaching the use of Programming…

Background: Analysing classroom dialogue is a widely used approach for understanding students' learning, often requiring team-based collaborative research. This presents a challenge for single researchers due to the labour-intensive nature of the process. Emerging advancements in large language models (LLMs) such as ChatGPT, enhance qualitative…

Worked examples, which present an explained code for solving typical programming problems are among the most popular types of learning content in programming classes. Most approaches and tools for presenting these examples to students are based on line-by-line explanations of the example code. However, instructors rarely have time to provide…

Text classification has allowed researchers to analyze natural language data at a previously impossible scale. However, a text classifier is only as valid as the the annotations on which it was trained. Further, the cost of training a classifier depends on annotators' ability to quickly and accurately apply the coding scheme to each text. Thus,…

Background: Formative assessments are needed to enable monitoring how student knowledge develops throughout a unit. Constructed response items which require learners to formulate their own free-text responses are well suited for testing their active knowledge. However, assessing such constructed responses in an automated fashion is a complex task…

Scholars have applied automatic content analysis to study computer-mediated communication in computer-supported collaborative learning (CSCL). Since CSCL also takes place in face-to-face interactions, we studied the automatic coding accuracy of manually transcribed face-to-face communication. We conducted our study in an authentic higher-education…