scrape
Fetch a single URL and return markdown content.
How to use it
scrape is exposed by the Crawl4ai MCP Server MCP server. Add the server to your MCP client (Claude Desktop, Cursor, Windsurf and others), and the scrape tool becomes available to the model automatically. See the full listing for setup details and every tool this server provides.
Install Crawl4ai MCP Server
pip install -r requirements.txtOther tools in Crawl4ai MCP Server (19)
Enable adaptive crawling (default: false)
Optional Crawl4AI browser config overrides
Multi-page breadth-first crawling with filtering and adaptive stopping.
Comprehensive site crawling with persistence (always requires outputdir).
Sitemap-based crawling with persistence (always requires outputdir).
Optional Crawl4AI crawler config overrides
(required): Starting URL for site crawl
Regex patterns to exclude URLs
Regex patterns URLs must match
Maximum link depth to follow (default: 1, max: 4)
Maximum sitemap entries to process (default: 1000)
Maximum pages to crawl (default: 5, max: 100)
(optional): If provided, persists content to disk and returns metadata only
Stay within the same domain (default: true)
Optional C4A-Script for page interaction
(required): Starting URL for the crawl
(required): URL to sitemap.xml
Request timeout (default: 45s, max: 600s)
(required): The URL to scrape