check_robots

MCP tool from Seo MCP by Mehmoodqureshi

Fetch & parse /robots.txt — UA groups, allow/disallow, declared sitemaps

How to use it

check_robots is exposed by the Seo MCP MCP server. Add the server to your MCP client (Claude Desktop, Cursor, Windsurf and others), and the check_robots tool becomes available to the model automatically. See the full listing for setup details and every tool this server provides.

Install Seo MCP

$npx -y @mehmoodqureshi/seo-mcp

FULL SEO MCP LISTING

Other tools in Seo MCP (6)

audit_page

Full on-page audit: title, meta description, canonical, robots, viewport, OG/Twitter, heading outline, word count, image alt coverage, link counts, structured-data presence → issues + 0–100 score. Pass pagespeed: true to also attach PageSpeed metrics

check_sitemap

Discover & summarize a sitemap (index vs urlset, URL count, sample)

extract_schema

Pull JSON-LD structured data and list schema.org @types (+ microdata, parse errors)

find_broken_links

HEAD-check links on a page; report 4xx/5xx/unreachable

keyword_ideas

Related queries for a seed term via Google Suggest (no volume)

pagespeed

Google PageSpeed Insights (Lighthouse): perf score, lab Core Web Vitals + real-user CrUX field data + top opportunities. Keyless (rate-limited); set PAGESPEEDAPIKEY to raise the quota