Web Bot Auth · Cloudflare Radar verified-bot registry · 2026-07-10

We lint every published bot key directory. 1 in 6 is broken or down.

79 of the 684 bots in Cloudflare's verified-bot registry publish a Web Bot Auth key directory — the /.well-known/http-message-signatures-directory a verifier reads to confirm a crawler is who it claims. We fetch and grade all of them with @sitedex/web-bot-auth-lint.

13/79
directories fail verification or are unreachable. Check yours before a verifier does: npx web-bot-auth-lint your-domain.com

All 79 directories, one square each

✓26 verify, zero findings !40 verify, with warnings ✕9 fail verification ~4 unreachable

The most common hygiene problems

Warning-grade findings across all 79 directories. A directory can carry several; warnings don't block verification today, but each is a mistake a stricter verifier could reject.

The 9 failures

BotOperatorWhy a verifier rejects it
Cloudflare Radar URL ScannerCloudflareReturned HTTP 404, not 200. The directory has no `keys` array.
ConsentCheck BotConsentCheckReturned HTTP 404, not 200. The response body is not valid JSON.
DNSnifferBotDNSnifferNo usable key in this directory — Invalid public-key value: key #1 (x is invalid bytes, expected 32).
FlowpaneFlowpane LtdNo key is usable yet — every key's not-before time is in the future: QZdoanWPKy-cHIy6B080FKtxV0Zky1tSDQ6WIslvi7E.
NostraNostraNo key is usable yet — every key's not-before time is in the future: 3vKgyEtYsNMyY8M-CDk8_HzkuZO22aLy_LlHEjtT3WI, Pav1sqEdDlQ9qYyCUqV7J5oT3Eh2aTabW5-Ii6OCHT0.
Novellum AI CrawlNovellumReturned HTTP 404, not 200. The response body is not valid JSON.
PureConsentBotPureConsentThe response body is not valid JSON.
TrustLink-VerificationAgentPearl Industry Network LLCThe signature tag is "web-bot-auth"; a directory-response signature MUST carry tag="http-message-signatures-directory".
YouBotYou.comReturned HTTP 403, not 200. The response body is not valid JSON.

Unreachable after 3 attempts with backoff: Amazon Bedrock AgentCore Browser (AP South) (attempt 3: ETIMEDOUT: fetch failed); Quartr Crawler (attempt 3: UND_ERR_CONNECT_TIMEOUT: fetch failed); sourcedash (attempt 3: EAI_AGAIN: fetch failed); TwinAgent (attempt 3: ENOTFOUND: fetch failed).