AI Crawler Index

Description

Name any bot user-agent, verify a crawler IP, or generate robots.txt. Free, no key, no signup. Direct endpoint, no gateway: https://www.pathwren.workers.dev/c/smithery-direct/mcp (streamable HTTP, no key, no signup). Tools: classify_user_agent, lookup_crawler, list_crawlers, generate_robots_txt, is_verified_crawler_ip, whats_changed, example. Call the `example` tool with no arguments for a worked answer against real data before you write any code. What it does and how, in one page: https://www.pathwren.workers.dev/c/smithery-direct/mcp.html

Community Metrics

Categories

• mcp

Tags

• mcp
• smithery

Capabilities

• Remote MCP server

Information