Crawl4ai MCP Server
未认领🕷️ A lightweight Model Context Protocol (MCP) server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents. Similar to Firecrawl's API but self-hosted and free. Perfect for integrating web scraping into your AI workflows with OpenAI Agents SDK, Cursor, Claude Code, and other MCP-compatible tools.
安装
pip install -r requirements.txt工具（20 个）
adaptive
Enable adaptive crawling (default: false)
browser
Optional Crawl4AI browser config overrides
crawl
Multi-page breadth-first crawling with filtering and adaptive stopping.
crawl_site
Comprehensive site crawling with persistence (always requires outputdir).
crawl_sitemap
Sitemap-based crawling with persistence (always requires outputdir).
crawler
Optional Crawl4AI crawler config overrides
entry_url
(required): Starting URL for site crawl
exclude_patterns
Regex patterns to exclude URLs
include_patterns
Regex patterns URLs must match
max_depth
Maximum link depth to follow (default: 1, max: 4)
max_entries
Maximum sitemap entries to process (default: 1000)
max_pages
Maximum pages to crawl (default: 5, max: 100)
output_dir
(optional): If provided, persists content to disk and returns metadata only
same_domain_only
Stay within the same domain (default: true)
scrape
Fetch a single URL and return markdown content.
script
Optional C4A-Script for page interaction
seed_url
(required): Starting URL for the crawl
sitemap_url
(required): URL to sitemap.xml
timeout_sec
Request timeout (default: 45s, max: 600s)
url
(required): The URL to scrape