Universal Web Scraper

AI-driven data extraction from 55+ Actors across all major platforms. This skill automatically selects the best Actor for your task.

Prerequisites

(No need to check it upfront)

```
.env
```
file with
```
APIFY_TOKEN
```
Node.js 20.6+ (for native
```
--env-file
```
support)
```
mcpc
```
CLI tool:
```
npm install -g @apify/mcpc
```

Workflow

Copy this checklist and track progress:

Task Progress:
- [ ] Step 1: Understand user goal and select Actor
- [ ] Step 2: Fetch Actor schema via mcpc
- [ ] Step 3: Ask user preferences (format, filename)
- [ ] Step 4: Run the scraper script
- [ ] Step 5: Summarize results and offer follow-ups

Step 1: Understand User Goal and Select Actor

First, understand what the user wants to achieve. Then select the best Actor from the options below.

Instagram Actors (12)

Actor ID	Best For
`apify/instagram-profile-scraper`	Profile data, follower counts, bio info
`apify/instagram-post-scraper`	Individual post details, engagement metrics
`apify/instagram-comment-scraper`	Comment extraction, sentiment analysis
`apify/instagram-hashtag-scraper`	Hashtag content, trending topics
`apify/instagram-hashtag-stats`	Hashtag performance metrics
`apify/instagram-reel-scraper`	Reels content and metrics
`apify/instagram-search-scraper`	Search users, places, hashtags
`apify/instagram-tagged-scraper`	Posts tagged with specific accounts
`apify/instagram-followers-count-scraper`	Follower count tracking
`apify/instagram-scraper`	Comprehensive Instagram data
`apify/instagram-api-scraper`	API-based Instagram access
`apify/export-instagram-comments-posts`	Bulk comment/post export

Facebook Actors (14)

Actor ID	Best For
`apify/facebook-pages-scraper`	Page data, metrics, contact info
`apify/facebook-page-contact-information`	Emails, phones, addresses from pages
`apify/facebook-posts-scraper`	Post content and engagement
`apify/facebook-comments-scraper`	Comment extraction
`apify/facebook-likes-scraper`	Reaction analysis
`apify/facebook-reviews-scraper`	Page reviews
`apify/facebook-groups-scraper`	Group content and members
`apify/facebook-events-scraper`	Event data
`apify/facebook-ads-scraper`	Ad creative and targeting
`apify/facebook-search-scraper`	Search results
`apify/facebook-reels-scraper`	Reels content
`apify/facebook-photos-scraper`	Photo extraction
`apify/facebook-marketplace-scraper`	Marketplace listings
`apify/facebook-followers-following-scraper`	Follower/following lists

TikTok Actors (14)

Actor ID	Best For
`clockworks/tiktok-scraper`	Comprehensive TikTok data
`clockworks/free-tiktok-scraper`	Free TikTok extraction
`clockworks/tiktok-profile-scraper`	Profile data
`clockworks/tiktok-video-scraper`	Video details and metrics
`clockworks/tiktok-comments-scraper`	Comment extraction
`clockworks/tiktok-followers-scraper`	Follower lists
`clockworks/tiktok-user-search-scraper`	Find users by keywords
`clockworks/tiktok-hashtag-scraper`	Hashtag content
`clockworks/tiktok-sound-scraper`	Trending sounds
`clockworks/tiktok-ads-scraper`	Ad content
`clockworks/tiktok-discover-scraper`	Discover page content
`clockworks/tiktok-explore-scraper`	Explore content
`clockworks/tiktok-trends-scraper`	Trending content
`clockworks/tiktok-live-scraper`	Live stream data

YouTube Actors (5)

Actor ID	Best For
`streamers/youtube-scraper`	Video data and metrics
`streamers/youtube-channel-scraper`	Channel information
`streamers/youtube-comments-scraper`	Comment extraction
`streamers/youtube-shorts-scraper`	Shorts content
`streamers/youtube-video-scraper-by-hashtag`	Videos by hashtag

Google Maps Actors (4)

Actor ID	Best For
`compass/crawler-google-places`	Business listings, ratings, contact info
`compass/google-maps-extractor`	Detailed business data
`compass/Google-Maps-Reviews-Scraper`	Review extraction
`poidata/google-maps-email-extractor`	Email discovery from listings

Other Actors (6)

Actor ID	Best For
`apify/google-search-scraper`	Google search results
`apify/google-trends-scraper`	Google Trends data
`voyager/booking-scraper`	Booking.com hotel data
`voyager/booking-reviews-scraper`	Booking.com reviews
`maxcopell/tripadvisor-reviews`	TripAdvisor reviews
`vdrmota/contact-info-scraper`	Contact enrichment from URLs

Actor Selection by Use Case

Use Case	Primary Actors
Lead Generation	`compass/crawler-google-places` , `poidata/google-maps-email-extractor` , `vdrmota/contact-info-scraper`
Influencer Discovery	`apify/instagram-profile-scraper` , `clockworks/tiktok-profile-scraper` , `streamers/youtube-channel-scraper`
Brand Monitoring	`apify/instagram-tagged-scraper` , `apify/instagram-hashtag-scraper` , `compass/Google-Maps-Reviews-Scraper`
Competitor Analysis	`apify/facebook-pages-scraper` , `apify/facebook-ads-scraper` , `apify/instagram-profile-scraper`
Content Analytics	`apify/instagram-post-scraper` , `clockworks/tiktok-scraper` , `streamers/youtube-scraper`
Trend Research	`apify/google-trends-scraper` , `clockworks/tiktok-trends-scraper` , `apify/instagram-hashtag-stats`
Review Analysis	`compass/Google-Maps-Reviews-Scraper` , `voyager/booking-reviews-scraper` , `maxcopell/tripadvisor-reviews`
Audience Analysis	`apify/instagram-followers-count-scraper` , `clockworks/tiktok-followers-scraper` , `apify/facebook-followers-following-scraper`

Multi-Actor Workflows

For complex tasks, chain multiple Actors:

Workflow	Step 1	Step 2
Lead enrichment	`compass/crawler-google-places` →	`vdrmota/contact-info-scraper`
Influencer vetting	`apify/instagram-profile-scraper` →	`apify/instagram-comment-scraper`
Competitor deep-dive	`apify/facebook-pages-scraper` →	`apify/facebook-posts-scraper`
Local business analysis	`compass/crawler-google-places` →	`compass/Google-Maps-Reviews-Scraper`

Can't Find a Suitable Actor?

If none of the Actors above match the user's request, search the Apify Store directly:

bash

export $(grep APIFY_TOKEN .env | xargs) && mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call search-actors keywords:="SEARCH_KEYWORDS" limit:=10 offset:=0 category:="" | jq -r '.content[0].text'

Replace

SEARCH_KEYWORDS

with 1-3 simple terms (e.g., "LinkedIn profiles", "Amazon products", "Twitter").

Step 2: Fetch Actor Schema

Fetch the Actor's input schema and details dynamically using mcpc:

bash

export $(grep APIFY_TOKEN .env | xargs) && mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call fetch-actor-details actor:="ACTOR_ID" | jq -r ".content"

Replace

ACTOR_ID

with the selected Actor (e.g.,

compass/crawler-google-places

This returns:

Actor description and README
Required and optional input parameters
Output fields (if available)

Step 3: Ask User Preferences

Before running, ask:

Output format:
- Quick answer - Display top few results in chat (no file saved)
- CSV - Full export with all fields
- JSON - Full export in JSON format
Number of results: Based on character of use case

Step 4: Run the Script

Quick answer (display in chat, no file):

bash

node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_actor.js \
  --actor "ACTOR_ID" \
  --input 'JSON_INPUT'

CSV:

bash

node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_actor.js \
  --actor "ACTOR_ID" \
  --input 'JSON_INPUT' \
  --output YYYY-MM-DD_OUTPUT_FILE.csv \
  --format csv

JSON:

bash

node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_actor.js \
  --actor "ACTOR_ID" \
  --input 'JSON_INPUT' \
  --output YYYY-MM-DD_OUTPUT_FILE.json \
  --format json

Step 5: Summarize Results and Offer Follow-ups

After completion, report:

Number of results found
File location and name
Key fields available
Suggested follow-up workflows based on results:

If User Got	Suggest Next
Business listings	Enrich with `vdrmota/contact-info-scraper` or get reviews
Influencer profiles	Analyze engagement with comment scrapers
Competitor pages	Deep-dive with post/ad scrapers
Trend data	Validate with platform-specific hashtag scrapers

Error Handling

APIFY_TOKEN not found

- Ask user to create

.env

with

APIFY_TOKEN=your_token

mcpc not found

- Ask user to install

npm install -g @apify/mcpc

Actor not found

- Check Actor ID spelling

Run FAILED

- Ask user to check Apify console link in error output

Timeout

- Reduce input size or increase

--timeout

apify-ultimate-scraper

NPX Install

Tags

SKILL.md Content

Universal Web Scraper

Prerequisites

Workflow

Step 1: Understand User Goal and Select Actor

Instagram Actors (12)

Facebook Actors (14)

TikTok Actors (14)

YouTube Actors (5)

Google Maps Actors (4)

Other Actors (6)

Actor Selection by Use Case

Multi-Actor Workflows

Can't Find a Suitable Actor?

Step 2: Fetch Actor Schema

Step 3: Ask User Preferences

Step 4: Run the Script

Step 5: Summarize Results and Offer Follow-ups

Error Handling