adaptive
Enable adaptive crawling (default: false)
How to use it
adaptive is exposed by the Crawl4ai MCP Server MCP server. Add the server to your MCP client (Claude Desktop, Cursor, Windsurf and others), and the adaptive tool becomes available to the model automatically. See the full listing for setup details and every tool this server provides.
Install Crawl4ai MCP Server
pip install -r requirements.txtOther tools in Crawl4ai MCP Server (19)
Optional Crawl4AI browser config overrides
Multi-page breadth-first crawling with filtering and adaptive stopping.
Comprehensive site crawling with persistence (always requires outputdir).
Sitemap-based crawling with persistence (always requires outputdir).
Optional Crawl4AI crawler config overrides
(required): Starting URL for site crawl
Regex patterns to exclude URLs
Regex patterns URLs must match
Maximum link depth to follow (default: 1, max: 4)
Maximum sitemap entries to process (default: 1000)
Maximum pages to crawl (default: 5, max: 100)
(optional): If provided, persists content to disk and returns metadata only
Stay within the same domain (default: true)
Fetch a single URL and return markdown content.
Optional C4A-Script for page interaction
(required): Starting URL for the crawl
(required): URL to sitemap.xml
Request timeout (default: 45s, max: 600s)
(required): The URL to scrape