NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1475801
Record Type: Journal
Publication Date: 2025-Aug
Pages: 51
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0049-1241
EISSN: EISSN-1552-8294
Available Date: 0000-00-00
Quantifying Narrative Similarity across Languages
Sociological Methods & Research, v54 n3 p933-983 2025
How can one understand the spread of ideas across text data? This is a key measurement problem in sociological inquiry, from the study of how interest groups shape media discourse, to the spread of policy across institutions, to the diffusion of organizational structures and institution themselves. To study how ideas and narratives diffuse across text, we must first develop a method to identify whether texts share the same information and narratives, rather than the same broad themes or exact features. We propose a novel approach to measure this quantity of interest, which we call "narrative similarity," by using large language models to distill texts to their core ideas and then compare the similarity of "claims" rather than of words, phrases, or sentences. The result is an estimand much closer to narrative similarity than what is possible with past relevant alternatives, including exact text reuse, which returns lexically similar documents; topic modeling, which returns topically similar documents; or an array of alternative approaches. We devise an approach to providing out-of-sample measures of performance (precision, recall, F1) and show that our approach outperforms relevant alternatives by a large margin. We apply our approach to an important case study: The spread of Russian claims about the development of a Ukrainian bioweapons program in U.S. mainstream and fringe news websites. While we focus on news in this application, our approach can be applied more broadly to the study of propaganda, misinformation, diffusion of policy and cultural objects, among other topics.
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Russia; Ukraine; United States
Grant or Contract Numbers: N/A
Author Affiliations: 1Department of Sociology, University of Oregon, Eugene, OR, USA; 2Center for Social Media and Politics, New York University, New York, NY, USA; 3Department of Political Science, University of Kansas, Lawrence, KS, USA; 4Department of Political Science, University of California San Diego, San Diego, CA, USA; 5Wilf Family Department of Politics, New York University New York, NY, USA; 6School of Information, University of Michigan, Ann Arbor, MI, USA; 7School of Politics, Security and International Affairs, University of Central Florida, Orlando, FL, USA