blog-scraper

Original：🇺🇸 English

Translated

1 scriptsChecked / no sensitive code detected

Scrape blog posts via RSS feeds (free, no API key) with Apify fallback for JS-heavy sites. Use when you need to monitor competitor blogs, track industry content, or aggregate blog posts by keyword.

2installs

Sourceathina-ai/goose-skills

Added on2026-03-27

NPX Install

npx skill4agent add athina-ai/goose-skills blog-scraper

SKILL.md Content

View Translation Comparison →

Blog Scraper

Scrape blog posts via RSS/Atom feeds (free) with optional Apify fallback for JS-heavy sites.

Quick Start

For RSS mode (free), only dependency is

pip install requests

. No API key needed.

bash

# Scrape a blog's RSS feed
python3 skills/blog-scraper/scripts/scrape_blogs.py \
  --urls "https://example.com/blog" --days 30

# Multiple blogs with keyword filter
python3 skills/blog-scraper/scripts/scrape_blogs.py \
  --urls "https://blog1.com,https://blog2.com" --keywords "AI,marketing" --output summary

# Force Apify for JS-heavy sites
python3 skills/blog-scraper/scripts/scrape_blogs.py \
  --urls "https://example.com" --mode apify

How It Works

Auto Mode (default)

For each URL, tries to discover an RSS/Atom feed:
- Checks HTML
```
<link rel="alternate">
```
  tags
- Probes common paths:
```
/feed
```
  ,
```
/rss
```
  ,
```
/atom.xml
```
  ,
```
/feed.xml
```
  ,
```
/rss.xml
```
  ,
```
/blog/feed
```
  ,
```
/index.xml
```
Parses discovered feeds (supports RSS 2.0 and Atom)
If any URLs fail, falls back to Apify
```
jupri/rss-xml-scraper
```
(if token available)
Applies date and keyword filtering client-side

RSS Mode

Only tries RSS feeds, no Apify fallback.

Apify Mode

Uses Apify actor directly, skipping RSS discovery.

CLI Reference

Flag	Default	Description
`--urls`	required	Blog URL(s), comma-separated
`--keywords`	none	Keywords to filter (comma-separated, OR logic)
`--days`	30	Only include posts from last N days
`--max-posts`	50	Max posts to return
`--mode`	auto	`auto` (RSS + fallback), `rss` (RSS only), `apify` (Apify only)
`--output`	json	Output format: `json` or `summary`
`--token`	env var	Apify token (only needed for Apify mode/fallback)
`--timeout`	300	Max seconds for Apify run

Cost

RSS mode: Free (no API, no tokens)
Apify mode: Uses
```
jupri/rss-xml-scraper
```
— minimal Apify credits