Over the past decade, social scientists have adapted computational methods for qualitative text analysis, with the hope that they can match the accuracy and reliability of hand coding. The emergence of GPT and open-source generative large language models (LLMs) has transformed this process by shifting from programming to engaging with models using…

Educators and researchers are interested in ways that ChatGPT and other generative AI tools might move beyond the role of "cheatbot" and become part of the network of resources students use for writing. We studied how high school students used ChatGPT as a writing support while writing arguments about topics like school mascots. We…

Linguistic phenomena (e.g., words and syntactic structure) co-occur with a wide variety of meanings. These systematic correlations can help readers to interpret a text and create predictions about upcoming material. However, to what extent these correlations influence discourse processing is still unknown. We address this question by examining…

Using a quasi-experimental method and content analysis as a technique, this study tests ChatGPT, in its version 4, by assessing its textual characteristics and overall understanding regarding the recognition criteria of provisions under International Accounting Standards (IAS) 37, as issued by the International Accounting Standards Board (IASB).…

Large Language Models (LLMs) present a radically new paradigm for the study of "information foraging behavior." We study how LLM technology is used for pedagogical content creation by a sample of 25 participants in a doctoral-level Artificial Intelligence (AI) in Education course, and the role of computational-thinking skills in shaping…

We conducted two experiments to assess the alignment between Generative AI (GenAI) text personalization and hypothetical readers' profiles. In Experiment 1, four LLMs (i.e., Claude 3.5 Sonnet; Llama; Gemini Pro 1.5; ChatGPT 4) were prompted to tailor 10 science texts (i.e., biology, chemistry, physics) to accommodate four different profiles…

This paper systematically explores how Large Language Models (LLMs) generate explanations of code examples of the type used in intro-to-programming courses. As we show, the nature of code explanations generated by LLMs varies considerably based on the wording of the prompt, the target code examples being explained, the programming language, the…

This paper describes the results of an archival desk-study that analyzed worksheets produced by four students using ChatGPT as a coach. ChatGPT is a Generative Pre-Trained Large Language model that can write comprehensively in various languages and styles. It was discovered that it could pass university level physics exams and perform at the level…

The value of generative AI tools in higher education has received considerable attention. Although there are many proponents of its value as a learning tool, many are concerned with the issues regarding academic integrity and its use by students to compose written assessments. This study evaluates and compares the output of three commonly used…

Presented study seeks to examine the potential applications of the OpenAI language model, GPT-3, within the realm of education. Specifically, the inquiry focuses on the feasibility of utilizing GPT-3 to generate essays based on customized prompts. To this end, the experimentation involved providing GPT-3 with tailored prompts derived from diverse…

Investigating links between temporal features of the writing process (e.g., bursts and pauses during writing) and the linguistic features found in written products would help us better understand intersections between the writing process and product. However, research on this topic is rare. This article illustrates a method to examine associations…

Synthesis writing is widely taught across domains and serves as an important means of assessing writing ability, text comprehension, and content learning. Synthesis writing differs from other types of writing in terms of both cognitive and task demands because it requires writers to integrate information across source materials. However, little is…

Large language models present new opportunities for teaching and learning. The response accuracy of these models, however, is believed to depend on the prompt quality which can be a challenge for students. In this study, we aimed to explore how undergraduate students use ChatGPT for problem-solving, what prompting strategies they develop, the link…

We report on a college-level study of student reflection and instructor prompts using scoring and corpus analysis methods. We collected 340 student reflections and 24 faculty prompts. Reflections were scored using trait and holistic scoring and then reflections and faculty prompts were analyzed using Natural Language Processing to identify…

Evaluating creativity of verbal responses or texts is a challenging task due to psychometric issues associated with subjective ratings and the peculiarities of textual data. We explore an approach to objectively assess the creativity of responses in a sentence generation task to (1) better understand what language-related aspects are valued by…