honeybot.ai
robots.txt

Robots Exclusion Standard data for honeybot.ai

Resource Scan

Scan Details

Site Domain honeybot.ai
Base Domain honeybot.ai
Scan Status Ok
Last Scan2026-03-07T06:14:50+00:00
Next Scan 2026-04-06T06:14:50+00:00

Last Scan

Scanned2026-03-07T06:14:50+00:00
URL https://honeybot.ai/robots.txt
Redirect https://www.honeybot.ai/robots.txt
Redirect Domain www.honeybot.ai
Redirect Base honeybot.ai
Domain IPs 104.26.0.119, 104.26.1.119, 172.67.74.188, 2606:4700:20::681a:177, 2606:4700:20::681a:77, 2606:4700:20::ac43:4abc
Redirect IPs 104.26.0.119, 104.26.1.119, 172.67.74.188, 2606:4700:20::681a:177, 2606:4700:20::681a:77, 2606:4700:20::ac43:4abc
Response IP 104.26.0.119
Found Yes
Hash 90071c1ec839671c0f27cdda072a50172e92a73a923b19fbf786068ac154042d
SimHash 6d102e16c081

Groups

*

Rule Path
Allow /
Allow /bots/
Allow /chats/
Allow /pricing/
Allow /faq/
Allow /blog/
Allow /affiliates/
Allow /support/
Allow /generate-image/
Allow /privacy/
Allow /policies/
Allow /terms-and-conditions/
Allow /create-your-honeybot/
Allow /sitemap.xml
Disallow /admin/
Disallow /accounts/
Disallow /complaint/
Disallow /my-honeybot-page/
Disallow /chat/
Disallow /api/
Disallow /media/private/
Disallow /static/private/
Disallow /cdn-cgi/

Other Records

Field Value
sitemap https://www.honeybot.ai/sitemap.xml
sitemap https://photos.honeybot.ai/sitemap.xml

Comments

  • Crawl-delay: 10