Robots.txt Policy Lint

Description

Lint your robots.txt and see which AI crawlers it really blocks. Free, no key, nothing to install. Direct endpoint, no gateway: https://www.pathwren.workers.dev/c/smithery-direct/mcp/robots (streamable HTTP, no key, no signup). Tools: lint_robots_txt, check_path_allowed, audit_ai_access, diff_robots_txt, merge_policy, example. Call the `example` tool with no arguments for a worked answer against real data before you write any code. What it does and how, in one page: https://www.pathwren.workers.dev/c/smithery-direct/mcp-robots.html

Community Metrics

Categories

• mcp

Tags

• mcp
• smithery

Capabilities

• Remote MCP server

Information