Abstract

Detecting organized political campaigns is of paramount importance in fighting against disinformation on social media. Existing approaches for the identification of such organized actions employ techniques mostly from network science, graph machine learning and natural language processing. Their ultimate goal is to analyze the relationships and interactions (e.g. re-posting) among users and the textual similarities of their posts. Despite their effectiveness in recognizing astroturf campaigns, these methods face significant challenges, notably the class imbalance in available training datasets. To mitigate this issue, recent methods usually resort to data augmentation or increasing the number of positive samples, which may not always be feasible or sufficient in real-world settings. Following a different path, in this paper, we propose a novel framework for identifying astroturf campaigns based solely on large language models (LLMs), introducing a Balanced Retrieval-Augmented Generation (Balanced RAG) component. Our approach first gives both textual information concerning the posts (in our case tweets) and the user interactions of the social network as input to a language model. Then, through prompt engineering and the proposed Balanced RAG method, it effectively detects coordinated disinformation campaigns on X (Twitter). The proposed framework does not require any training or fine-tuning of the language model. Instead, by strategically harnessing the strengths of prompt engineering and Balanced RAG, it facilitates LLMs to overcome the effects of class imbalance and effectively identify coordinated political campaigns. The …

Metadata

publication

WWW'25 Companion Proceedings (BeyondFacts'25), 2025

year

2025

publication date

2025/1/21

authors

Nikos Kanakaris, Heng Ping, Xiongye Xiao, Nesreen K Ahmed, Luca Luceri, Emilio Ferrara, Paul Bogdan

link

https://arxiv.org/abs/2501.11849

resource_link

https://arxiv.org/pdf/2501.11849

conference

WWW'25 Companion Proceedings (BeyondFacts'25)

publisher

arXiv preprint arXiv:2501.11849