Extracting Intelligence from RSS News Feeds Using Python and AI - From Global Headlines to Actionable Intelligence (Chet Hosmer)（Z-Library）

Page 1

Extracting Intelligence from RSS News Feeds Using Python and AI From Global Headlines to Actionable Intelligence — Chet Hosmer

Page 2

Extracting Intelligence from RSS News Feeds Using Python and AI From Global Headlines to Actionable Intelligence Chet Hosmer

Page 3

Extracting Intelligence from RSS News Feeds Using Python and AI: From Global Headlines to Actionable Intelligence ISBN-13 (pbk): 979-8-8688-2772-3 ISBN-13 (electronic): 979-8-8688-2773-0 https://doi.org/10.1007/979-8-8688-2773-0 Copyright © 2026 by Chet Hosmer This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Managing Director, Apress Media LLC: Welmoed Spahr Acquisitions Editor: Susan McDermott Project Manager: Jessica Vakili Cover designed by eStudioCalamar Cover image designed by Pixabay Distributed to the book trade worldwide by Springer Science+Business Media New York, 1 New York Plaza, New York, NY 10004. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is a Delaware LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation. For information on translations, please e-mail booktranslations@springernature.com; for reprint, paperback, or audio rights, please e-mail bookpermissions@springernature.com. Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales web page at http://www.apress.com/bulk-sales. If disposing of this product, please recycle the paper Chet Hosmer Longs, SC, USA

Page 4

This book is dedicated to the outstanding team at the College of Applied Science and Technology (CAST) at the University of Arizona. Your support, encouragement, and leadership in cybersecurity education continue to inspire both students and colleagues alike. I am deeply grateful to Jason, Tracy, Haley, Jake, Carla, Marnie, Angela, and Joey, as well as the entire SOC and FM team. Your dedication to innovation, collaboration, and preparing the next generation of cybersecurity professionals has made a lasting impact. Thank you for your commitment, your passion, and for fostering an environment where curiosity, learning, and discovery thrive.

Page 5

v Table of Contents About the Author xi About the Technical Reviewer xiii Acknowledgments xv Preface xvii Foreword xix Introduction xxi Chapter 1: Understanding RSS Feed Formats 1 Anatomy of an RSS Feed1 High-Level Structure of an RSS Feed 2 Simple RSS Example 3 Explanation of Each Element of the RSS Example 4 What Makes This Such a Good RSS XML Example? 6 Chapter 2: Accessing and Parsing RSS Feeds with Python 9 What Is Feedparser and How Do I Install It? 10 Installing Feedparser into Your Python Environment 10 Sample Python Script That Uses Feedparser 11 Step-by-Step Explanation of the Feedparser Sample Script 12 Summary16

Page 6

vi Chapter 3: Processing RSS Data for AI Analysis 17 Obtaining an OpenAI API Key17 New Python Script with OpenAI and Feedparser 19 Script Part 1 20 Script Part 2 23 Script Extracting Article Content 24 Script Output 27 Adding OpenAI to Our Script 27 Final Script 30 Final Output: PrettyTable 34 Summary35 Chapter 4: Text Extraction and Multilingual Processing 37 Translating and Normalizing Non-English Content Using AI 38 Script-Chapter 4-1 39 Extracting Linguistic Signals from Native- Language Text 44 Building Author Profiles from RSS Feeds Using AI 49 Script Excerpt 50 Script Output 53 Summary54 Chapter 5: Extracting Actionable Intelligence: Name Entity Recognition 57 Why Name Entity Recognition Matters: From Text to Intelligence Signals58 The Media Reporter: Informing an Audience 58 The Digital Forensic Investigator: Establishing Facts and Context 58 The Intelligence Analyst: Revealing Hidden Structure 59 Why This Distinction Matters 60 Table of ConTenTs

Page 7

vii Understanding Entities in Real-World Reporting 60 People: Beyond the Author 60 Organizations: Power, Influence, and Alignment 61 Locations: Geography As Signal 62 From Mentions to Meaning 62 Why Consistent Extraction Matters More Than Perfect Accuracy 63 The Reality of Imperfect Data 63 Leveraging AI-Assisted NER with Python by Crafting Prompts for Entity Extraction 63 Implementing Entity Extraction in Python 64 Example: NER Identification (Extraction-First) 65 Developing an AI-Enhanced Python Script 66 Main Script Execution Flow 67 Sample Output from Script 5-1 68 Building the NER Extraction Prompt 70 Parsing and Structuring NER Results 72 Interpreting Entity Results 74 Architectural Flow of Script 5-2 76 Conceptual Shift: From Article to Entity Network 81 Foreign RSS Feed Example 81 Limitations, Bias, and Validation 85 Summary85 Looking Ahead to Chapter 6: From Entities to Deeper Sentiment and Threat Analysis 87 Chapter 6: Sentiment and Threat Analysis in OSINT 89 Sentiment Analysis in OSINT 90 What Is Sentiment Analysis? 90 Why Sentiment Analysis Matters in OSINT 91 Table of ConTenTs

Page 8

viii Analyzing Sentiment in RSS Articles: Basic Approach 91 Script 6-1 Analyzing Sentiment 92 Sentiment Script 6-1 Sample Output 96 English Article 96 Sentiment Script 6-1 Sample Output 97 German Article 97 What Are the Risks and Limitations of Sentiment Analysis 98 Threat Analysis in OSINT 99 What Is Threat Analysis? 99 Why Threat Analysis Is Critical to OSINT 99 Analyzing Threats in RSS Articles—Basic Approach 100 Script 6-2 Analyzing Threat 100 Threat Script 6-2 Sample Output 103 Threat Script 6-2 Sample Output 104 Differentiating Sentiment vs Threat 104 Chapter Implications 105 Sentiment and Threat Analysis in OSINT 105 Sentiment Analysis As an Intelligence Signal 105 Threat Analysis As Operational Intelligence 107 Overall Impact of Chapter 6 109 What to Expect in Chapter 7110 Chapter 7: Agentic AI for Multi-Feed Intelligence Processing 113 What Is Agentic AI and How Does It Differ from Prompt-Driven Analysis 114 Prompt-Driven Analysis: Structured but Reactive 115 Script 7-1 Agentic AI 119 Script Objective Statement 120 Examining the Main Loop 121 Examining the Results 123 Table of ConTenTs

Page 9

ix Script 7-2 Agentic AI Enhanced124 Main Loop 7-2 124 Examining the Results from Script 7-2126 Summary127 Chapter Challenge: Enhancing the Agent’s Objective 128 Suggested Exercise 130 Next Steps 130 Chapter 8: Real-World Applications and Case Studies 131 Space Exploration Agentic AI OSINT 132 Script-Chapter 8-SpaceResearchpy 134 Climate Change Agentic AI OSINT 137 Summary140 Chapter 9: What Lies Ahead? 142 Chapter 9: Future Agentic AI RSS Feed Analysis 143 From Static Analysis to Continuous Intelligence 144 Autonomous Discovery of Relevant Information144 Continuous Agentic Processing 145 Intelligent Alerting and Notification 146 The Role of Human Oversight 146 Python As the Foundation of Intelligent Analysis 147 Final Thoughts 147 Appendix 149 Index 161 Table of ConTenTs

Page 10

xi Chet Hosmer is the founder of Python Forensics, a non-profit organization that provides research and Python scripts to help with advanced investigative challenges. Chet also serves as a Designated Campus Colleague at the University of Arizona. Chet has made numerous appearances to discuss emerging cyber threats including NPR, ABC News, Forbes, IEEE, The New York Times, The Washington Post, Government Computer News, Salon.com, and Wired magazine. He has seven published books with Apress and Elsevier that focus on Python Forensics, data hiding, passive network defense strategies, PowerShell, and IoT. In addition, Chet presents at major conferences each year including RSA, TechnoSecurity, HTCIA, Blackhat, and DEFCON. About the Author

Page 11

xiii Dr. Gary C. Kessler, CISSP, is president and janitor of Gary Kessler Associates, a training, research, and consulting company specializing in maritime cybersecurity. Gary holds a B.A. in Mathematics, an M.S. in Computer Science, and a Ph.D. in Computing Technology in Education and has been in the information security field since the late 1970s. Co-author of the book Maritime Cybersecurity, 2nd edition, and author of dozens of papers on technology-related topics, he is a retired professor of cybersecurity with a research interest in the Automatic Identification System (AIS). An international lecturer, Gary is a guest faculty member at the US Coast Guard Academy, on the advisory board of Cydome, instructor and mentor at the CyberBoat Challenge, a co-founder of the Maritime Hacking Village, and a Fellow in the USCG Auxiliary Cybersecurity Directorate. Gary is also a SCUBA instructor and holds a 50 GT Merchant Mariner Credential. More information can be found at https://urldefense.com/v3/__https://www.garykessler.net__; !!NLFGqXoFfo8MMQ!rXOXyNw7qdngSpZxFvQzMOVp81dJF5CYg2aSzyjrpRB hUvJ595WDAOU2NMzpyRh7YARTgoFfbfNs4Oe6CvE$. About the Technical Reviewer

Page 12

xv Acknowledgments My wife Janet, for your unwavering support, encouragement, and belief in me every single day. Dr. Gary Kessler for his incredible technical editing of this book. I’m deeply indebted for your assistance in making this book better chapter by chapter. Mike Raggo, for your constant support even in the early days of this project. Your insights into this field and this genre are truly invaluable. Greg Kipper, for helping shape this book with your thoughtful insights and your visionary view of the future. Amber Schroader, for always saving me a place to speak at PFIC events. These conferences are consistently on the leading edge of digital forensics and cybersecurity. OpenAI, for developing large language models that have transformed how we interact with knowledge and making these capabilities broadly accessible. Guido van Rossum, for creating the Python programming language, which continues to evolve and empower innovation across computer science and beyond. Julie Lewis, for the opportunity to share the stage with you and the incredible team at Digital Mountain. Your leadership in digital investigations continues to make a significant impact. Allison Dowd and the Techno Security team, for always welcoming me to speak and for holding a conference that continues to grow and inspire the cybersecurity community.

Page 13

xvi Kevin DeLong, for your continued support and for creating such a vibrant and welcoming community at the Cyber Social Hub. This book reflects the remarkable people and communities dedicated to advancing cybersecurity, digital investigation, intelligence analysis, and education. I am grateful to be part of such an inspiring field. aCknowledgmenTs

Page 14

xvii Preface We live in an age of unprecedented information availability. Every minute, thousands of articles, reports, and observations are published across the Internet, covering topics ranging from cybersecurity threats and geopolitical developments to scientific discoveries and technological innovation. While this abundance of information has tremendous value, it also presents a fundamental challenge: how do we efficiently identify the information that truly matters? For many professionals working in fields such as cybersecurity, intelligence analysis, research, and journalism, the problem is not the lack of information; it is the overwhelming volume of it. Important insights are often buried within massive streams of news articles, blog posts, and technical reports that are continuously being published across the world. One of the most reliable and structured ways to access this global information stream is through RSS feeds. For decades, RSS has provided a standardized method for publishing updates from trusted sources. Despite its simplicity and reliability, RSS remains an underutilized tool for large- scale information analysis. At the same time, advances in artificial intelligence and natural language processing have created entirely new opportunities for analyzing written content. Large language models can interpret text, extracting meaning, identifying entities, evaluating sentiment, and recognizing patterns across vast collections of documents. This book brings these two powerful capabilities together.

Page 15

xviii Using Python as the integration platform, we will build a practical system capable of retrieving RSS feeds, extracting article content, normalizing multilingual information, identifying key entities, analyzing sentiment and potential threats, and ultimately applying Agentic AI techniques to determine which information deserves further attention. Rather than presenting abstract theory, this book focuses on practical implementation. Each chapter introduces real Python scripts that demonstrate how these techniques can be applied to transform raw feed data into meaningful insights. By the end of the book, readers will have a working framework capable of turning large volumes of global information into structured intelligence. The techniques presented here are especially valuable for professionals involved in open source intelligence (OSINT), cybersecurity monitoring, research, and investigative analysis, but the underlying methods can be applied to virtually any domain where understanding large streams of textual information is important. The goal of this book is simple: to demonstrate how modern AI tools, when combined with Python and structured information sources, can dramatically improve our ability to discover meaningful insights within the global flow of information. In short, this book is about moving from raw information feeds to real intelligence. PrefaCe

Page 16

xix Foreword It is both a pleasure and an honor to write the foreword for this book. I have known Chet for more than twenty years, and the work I have seen him do and sometimes had the good fortune to be part of has always been interesting, innovative, and above all, useful. This project is no different. In this book, you will discover that the next big step in open source intelligence will not come from finding new data; it will come from rediscovering the best sources that are already out there and putting them to work. In this age of generative AI, decentralized communication, and ambient information, Chet shows that RSS feeds remain one of the most underappreciated sources of structured intelligence. As a career futurist and technologist, I have often spoken about how information would evolve to where it could adapt, learn, and collaborate with people. The methods outlined in these pages represent a tangible step toward that future. By using Python and data-curating techniques, this book illustrates how decades-old RSS feeds can feed quality data into an intelligent ecosystem capable of self-directed analysis and foresight for almost any problem. It also shows that by pairing the clarity of RSS with the interpretive power of artificial intelligence, we can build systems that do more than monitor headlines—they synthesize context, surface anomalies, and anticipate change. In essence, it is a guide to constructing digital sentinels that observe reality in real time. What makes this book especially valuable is its grounding in trust and structure. In a time when misinformation spreads virally through unverified social media platforms, the need for trustworthy, structured, and verifiable data has never been greater. Through RSS feeds, we gain direct access to curated, fact-checked data that stands up to analytical

Page 17

xx scrutiny. When combined with large language models and AI agents, these feeds become a dynamic network of global awareness, producing quality insights and actionable intelligence. For those who actively look for useful innovations, this work is both a toolkit and a guidebook. I'm excited for you and what you are about to learn in these pages. —Greg Kipper foreword

Page 18

xxi Introduction This research grew out of a simple realization: the world is overflowing with RSS feeds (Real-Simple-Syndication) and millions of them, yet only a fraction are actively mined for meaningful insights. Using Python and artificial intelligence, even these overlooked streams of information can be converted into actionable intelligence. And while no authoritative global count exists, web analysis platforms such as BuiltWith1 have estimated that more than 36 million websites publish RSS feeds. Why Choose RSS Feeds over Other Sources of Content? When building an automated intelligence-extraction pipeline, the quality and integrity of the underlying data source are fundamental. RSS feeds offer a unique advantage because they deliver professionally produced, fact-checked, and consistently structured information directly from established publishers. Each feed entry typically includes a well-formed title, author, timestamp, summary, category, and link to the full article. This uniformity dramatically reduces preprocessing overhead and ensures that downstream AI models receive clean, context-rich, and high-signal content that is ideal for summarization, translation, sentiment analysis, and topic classification. 1 RSS usage estimates are based on data provided by BuiltWith, a web technology analysis service (builtwith.com).

Page 19

xxii In contrast, platforms such as Twitter (X) and Reddit are dominated by short-form, user-generated content that varies widely in grammar, structure, credibility, and intent. Tweets, often emotional, sarcastic, or loose observations, lack the context necessary for reliable interpretation, and most importantly both platforms are heavily polluted by bots, spam, and coordinated mis- and disinformation. Reddit posts can offer deeper discussions, but they remain informal, conversational, and influenced by community dynamics rather than journalistic standards. In both cases, metadata is inconsistent, authors are frequently anonymous, and content often requires extensive cleaning before it becomes usable. For these reasons, RSS feeds serve as a fertile platform for extracting meaningful and actionable intelligence. Their structured format, editorial reliability, and low noise level make them exceptionally well-suited for automated analysis pipelines, especially when combined with modern tools, algorithms, and techniques. It is important to note that AI intelligent analyses thrive on high-quality input. While social media platforms can still contribute supplemental signals such as early indications of emerging events, RSS feeds remain the most stable, trustworthy, and analytically valuable source for building a robust intelligence-processing workflow. This overwhelming number of available RSS feeds creates fundamental challenges: Identifying high-value RSS feeds aligned with a specific domain or topic Programmatically extracting article metadata and content using Python libraries such as feedparser and Newspaper3k Handling multi-entry feeds and selecting articles most relevant to your objectives Integrating Python with AI models and well- designed prompts to distill the essential insights from curated feed content InTroduCTIon

Page 20

xxiii Our approach is to bridge the gap between raw RSS data and actionable, AI-driven insight. While millions of websites expose RSS feeds, the challenge lies in identifying the right sources, extracting high-quality content, and transforming that content into meaningful intelligence. Python, combined with modern AI models, provides an ideal platform to accomplish this. The goal is to give you a complete workflow from feed discovery and parsing to advanced analysis and real-world automation so you can build applications that understand, interpret, and react to the world’s information streams. To guide you through this process, the book is organized as follows. Chapters 1 and 2 Chapters 1 and 2 provide the basis to understand and process RSS feeds. They build the technical foundation by introducing the structure, anatomy, and variations of RSS feeds. You will learn how RSS is encoded, how metadata is represented, and why many real-world feeds do not strictly follow standards. With this background, we introduce practical Python techniques for retrieving and parsing feeds, handling failures and encoding issues, and converting XML into usable Python data structures. By the end of this section, you will have the essential tools needed to reliably ingest RSS feeds at scale. Chapters 3 and 4 Chapters 3 and 4 focus on preparing and processing feed content. Before meaningful analysis can begin, raw feed data must be normalized, cleaned, and enriched. These chapters focus on preparing RSS content for AI processing. You will learn how to detect and handle missing fields, remove unnecessary formatting, manage multilingual content, and extract full-text articles from linked HTML pages. These chapters show how InTroduCTIon