Automatically collect, analyze, and store industry news articles with intelligent filtering and dual-database storage
Automatically collect, analyze, and store industry news articles with intelligent filtering and dual-database storage 🎯 What This Workflow Does This comprehensive news scraping automation monitors multiple industry sources every 6 hours, intelligently extracts relevant content, analyzes it for keyword relevance, and stores high-quality articles in both a content library and knowledge base. Perfect for content marketers, researchers, industry analysts, and anyone building an AI-powered knowledge system. ✨ Key Features Multi-Source Aggregation: Collects from RSS feeds, Google News, and direct website scraping Smart Content Extraction: Uses Cheerio to parse HTML and extract clean article content AI-Ready Keyword Analysis: Automatically identifies and tags industry-specific terms Intelligent Filtering: Only stores articles meeting your relevance threshold (default: 30%) Dual Storage System: Saves to both content library (marketing) and knowledge base (AI training) Rate Limiting: Includes polite delays to respect website resources Fully Automated: Runs on schedule without manual intervention 🔧 Technical Highlights Schedule-based trigger (every 6 hours, customizable) Handles 3 content types: RSS feeds, Google News RSS, and web pages Limits to top 10 articles per source to manage volume Tracks 17+ insurance-specific keywords (easily customizable for any industry) Includes comprehensive error handling and timeout management Supabase integration (works with any REST API database) 📚 Perfect For Building AI chatbot knowledge bases Content marketing research Competitive intelligence gathering Industry trend monitoring News aggregation platforms SEO content research Market analysis and reporting 🚀 Setup Requirements n8n instance (self-hosted or cloud) Supabase account (free tier works) or any REST API database Basic understanding of JSON and HTTP requests All setup instructions included in sticky notes within the workflow! 📦 What's Included Complete workflow JSON (ready to import) 8 detailed sticky notes with beginner-friendly explanations Setup guide with step-by-step instructions Customization tips for different industries Example sources for insurance industry (easily adaptable) 🎨 Customization Options Modify schedule interval (hourly, daily, weekly) Add your own RSS feeds and websites Adjust relevance threshold Customize keyword lists for your industry Change article limits per source Modify storage destinations ⚠️ Important Notes Remember to update Supabase credentials before running Test with a single source before deploying all sources Respect website terms of service and robots.txt Consider API rate limits for your sources The workflow includes rate limiting to be respectful to websites 🏷️ Tags content-scraping rss-feeds web-scraping automation knowledge-base supabase cheerio news-aggregation content-marketing ai-training-data Version: 2.0 Last Updated: January 2025 Difficulty: Intermediate Estimated Setup Time: 30 minutes. An n8n automation workflow template by Shelly-Ann Davy.
Shelly-Ann Davy
web
paid
Others in the same category, ranked by how often they are opened.
