robots_txt

MCP tool from MCP Retrieval by Role1776

bool

How to use it

robots_txt is exposed by the MCP Retrieval MCP server. Add the server to your MCP client (Claude Desktop, Cursor, Windsurf and others), and the robots_txt tool becomes available to the model automatically. See the full listing for setup details and every tool this server provides.

Install MCP Retrieval

$go install github.com/Role1776/mcp-retrieval/app/cmd/mcp-retrieval@latest # needs Go 1.25.5+

FULL MCP RETRIEVAL LISTING

Other tools in MCP Retrieval (11)

string

int

int

int