crawl_site

MCP tool from Screaming Frog MCP by bzsasson

Start a headless background crawl (see note below)

How to use it

crawl_site is exposed by the Screaming Frog MCP MCP server. Add the server to your MCP client (Claude Desktop, Cursor, Windsurf and others), and the crawl_site tool becomes available to the model automatically. See the full listing for setup details and every tool this server provides.

Install Screaming Frog MCP

$uv tool install screaming-frog-mcp

FULL SCREAMING FROG MCP LISTING

Other tools in Screaming Frog MCP (8)

aggregate_crawl_data

Counts and group-by breakdowns over exported data ("how many 404s", "status code distribution") without reading rows into context

crawl_status

Check progress of a running crawl

delete_crawl

Permanently delete a crawl from the database

export_crawl

Export crawl data as CSV files (many export options available)