Crawl4ai MCP Server

Unclaimed

by sadiuysal

🕷️ A lightweight Model Context Protocol (MCP) server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents. Similar to Firecrawl's API but self-hosted and free. Perfect for integrating web scraping into your AI workflows with OpenAI Agents SDK, Cursor, Claude Code, and other MCP-compatible tools.

Install

$pip install -r requirements.txt

Unclaimed listing

Is this your MCP server?

This listing was auto-indexed from the public record. Claim it to edit the page, set compatibility and unlock growth tools. Takes under two minutes.

Claim this server

Tools (20)

adaptive

Enable adaptive crawling (default: false)

browser

Optional Crawl4AI browser config overrides

crawl

Multi-page breadth-first crawling with filtering and adaptive stopping.

crawl_site

Comprehensive site crawling with persistence (always requires outputdir).

crawl_sitemap

Sitemap-based crawling with persistence (always requires outputdir).

crawler

Optional Crawl4AI crawler config overrides

entry_url

(required): Starting URL for site crawl

exclude_patterns

Regex patterns to exclude URLs

include_patterns

Regex patterns URLs must match

max_depth

Maximum link depth to follow (default: 1, max: 4)

max_entries

Maximum sitemap entries to process (default: 1000)

max_pages

Maximum pages to crawl (default: 5, max: 100)

output_dir

(optional): If provided, persists content to disk and returns metadata only

same_domain_only

Stay within the same domain (default: true)

scrape

Fetch a single URL and return markdown content.

script

Optional C4A-Script for page interaction

seed_url

(required): Starting URL for the crawl

sitemap_url

(required): URL to sitemap.xml

timeout_sec

Request timeout (default: 45s, max: 600s)

url

(required): The URL to scrape