create_crawl

MCP tool from Olostep MCP Server by olostep

Start an async crawl that autonomously discovers and scrapes entire websites by following links. Returns a crawlid — the crawl runs in the background and does not return content in this response. You must then call getcrawlresults with the crawlid to poll status and retrieve the scraped pages (same

How to use it

Project documentation lists create_crawl for the Olostep MCP Server MCP server. Add the server to your MCP client (Claude Desktop, Cursor, Windsurf and others), then check which tools your installed version makes available. Tool availability can depend on configuration and credentials. A server handshake does not verify this tool’s behavior. See the full listing for setup details.

Install Olostep MCP Server

$docker run -i --rm -e OLOSTEP_API_KEY="your-api-key" olostep/mcp-server:local
FULL OLOSTEP MCP SERVER LISTING

Other tools in Olostep MCP Server (13)

answer_id

, object, task, result (JSON if provided), sources, created

answers

Search the web and return AI-powered answers in the JSON structure you want, with sources and citations.

batch_id

, status, totalurls, createdat, formats, country, parser, urls

batch_scrape_urls

Scrape up to 10k URLs at the same time. Perfect for large-scale data extraction.

crawl_id

, object, status, starturl, maxpages, created, formats, country, parser

create_map

Get all URLs on a website. Extract all URLs for discovery and analysis.

get_batch_results

Retrieve the results of a previously submitted batch scrape job using its batchid.

get_crawl_results

Retrieve the status and scraped pages for an async crawl started with createcrawl. This is the required companion to createcrawl — createcrawl only kicks off the job and returns a crawlid; this tool is how you actually fetch the discovered pages and their content.

get_webpage_content

Retrieves webpage content in clean markdown format with support for JavaScript rendering.

get_website_urls

Search and retrieve relevant URLs from a website, sorted by relevance to your query.

map_id

, object, url, totalurls, urls, searchquery, topn

scrape_website

Extract content from a single URL. Supports multiple formats and JavaScript rendering.

search_web

Search the Web for a given query and get structured results (non-AI, parser-based).