73% of the 121 sites we scanned haven’t set any AI rules yet.
We fetched the real robots.txt of 121 well-known independent US sites.
They haven’t chosen — not “open”, just undecided. Is your site one of them?
Scan it live →
· Set your rules in 1 step →
🟠 Unspecified — no explicit AI rules yet🔵 Managing — some AI crawlers have explicit rules⚫ Blocking — most AI crawlers blocked
The sites
Who's exposed, who's in control.
Site
Category
Posture
AI crawlers blocked
Method: live robots.txt fetch, parsed for 16 known AI crawler user-agents.
Posture reflects declared policy only — not actual AI traffic. Site owners can
claim and verify their listing.