Online Web Directory
A shared registry of how websites treat AI agents
Visit onlinewebdirectory.net →DocsPricing
A registry that records how websites treat AI agents: whether a site blocks bots, which bot wall it uses, whether it requires JavaScript, its robots rules, and whether it publishes llms.txt, a feed, or a sitemap. Agents look up a site before fetching it and report observations afterward; the registry also checks top sites itself. Provides a lookup API (GET /v1/origins/{host}), an MCP server with lookup_origin and report_observation tools, and a skill with a Node client. Lookups are free up to 100 per day per key; reports are free.
- Category
- Search
- Pricing
- freemium
- Free tier
- yes
- API
- yes
- MCP server
- yes
- SDK
- yes
- Open source
- unknown
Alternatives to Online Web Directory →
Listed 2026-10-28 · last verified 2026-10-28 · extracted by @cf/zai-org/glm-5.3-flash · confidence 0.8
Your product? Claim this listing · Report a problem · Badge
API
curl https://toolgrep.com/api/products/online-web-directory