Cloze items are a foundational approach to assessing readability. However, they require human data collection, thus making them impractical in automated metrics. The present study revisits the idea of assessing readability with cloze items and compares human cloze scores and readability judgments with predictions made by T5, a popular deep…

Worked examples, which present an explained code for solving typical programming problems are among the most popular types of learning content in programming classes. Most approaches and tools for presenting these examples to students are based on line-by-line explanations of the example code. However, instructors rarely have time to provide…

The ability to objectively quantify the complexity of a text can be a useful indicator of how likely learners of a given level will comprehend it. Before creating more complex models of assessing text difficulty, the basic building block of a text consists of words and, inherently, its overall difficulty is greatly influenced by the complexity of…

Readers often struggle to identify the main ideas in expository texts. Existing research and instruction provide some guidance on how to encourage readers to identify main ideas. However, there is substantial variability in how main ideas are operationalized and how readers are prompted to identify main ideas. This variability hinders…

For decades, educators have relied on readability metrics that tend to oversimplify dimensions of text difficulty. This study examines the potential of applying advanced artificial intelligence methods to the educational problem of assessing text difficulty. The combination of hierarchical machine learning and natural language processing (NLP) is…

We compared high school students' performance in a traditional comprehension assessment requiring them to identify key information and draw inferences from single texts, and a scenario-based assessment (SBA) requiring them to integrate, evaluate and apply information across multiple sources. Both assessments focused on a non-academic topic.…

This paper systematically explores how Large Language Models (LLMs) generate explanations of code examples of the type used in intro-to-programming courses. As we show, the nature of code explanations generated by LLMs varies considerably based on the wording of the prompt, the target code examples being explained, the programming language, the…

While hierarchical machine learning approaches have been used to classify texts into different content areas, this approach has, to our knowledge, not been used in the automated assessment of text difficulty. This study compared the accuracy of four classification machine learning approaches (flat, one-vs-one, one-vs-all, and hierarchical) using…

Online patient portals that facilitate communication between patient and provider can improve patients' medication adherence and health outcomes. The effectiveness of such web-based communication measures can be influenced by the health literacy (HL) of a patient. In the context of diabetes, low HL is associated with severe hypoglycemia and high…

Background knowledge is a strong predictor of reading comprehension; yet little is known about how different types of background knowledge affect comprehension. The study investigated the impacts of both domain and topic-specific background knowledge on students' ability to comprehend and learn from science texts. High school students (n = 3650)…

Inferential comprehension is necessary to connect ideas in a text together in a meaningful manner. There have been multiple studies on inferential comprehension involving texts of different genres (narrative and expository), but not a coherent overview of the findings of inferential comprehension by genre. The purpose of this study is to provide a…

We employed self-paced reading and event-related potential measures to investigate how adults of varying literacy levels use sentence context information when reading. Community-dwelling participants read strongly and weakly constraining sentences that ended with expected or unex- pected target words. Skilled readers showed N400s that were graded…

Text complexity has received increased attention due to the CCSS, which call for students to comprehend increasingly more complex texts as they progress through grades. Quantitative text complexity (or readability) indices are based on text attributes (e.g., sentence lengths, and lexical, syntactic, & semantic difficulty), quantified by…

Objective: To develop novel, scalable, and valid literacy profiles for identifying limited health literacy patients by harnessing natural language processing. Data Source: With respect to the linguistic content, we analyzed 283 216 secure messages sent by 6941 diabetes patients to physicians within an integrated system's electronic portal.…

Modern communication between health care professionals and patients increasingly relies upon secure messages (SMs) exchanged through an electronic patient portal. Despite the convenience of secure messaging, challenges include gaps between physician and patient expertise along with the asynchronous nature of such communication. Importantly, less…