Log In Sign Up

beano.com
robots.txt

Robots Exclusion Standard data for beano.com

Resource Scan

Scan Details

Site Domain beano.com
Base Domain beano.com
Scan Status Ok
Last Scan2025-03-08T23:23:27+00:00
Next Scan 2025-03-15T23:23:27+00:00

Last Scan

Scanned2025-03-08T23:23:27+00:00
URL https://beano.com/robots.txt
Redirect https://www.beano.com/robots.txt
Redirect Domain www.beano.com
Redirect Base beano.com
Domain IPs 104.26.14.165, 104.26.15.165, 172.67.72.182, 2606:4700:20::681a:ea5, 2606:4700:20::681a:fa5, 2606:4700:20::ac43:48b6
Redirect IPs 104.26.14.165, 104.26.15.165, 172.67.72.182, 2606:4700:20::681a:ea5, 2606:4700:20::681a:fa5, 2606:4700:20::ac43:48b6
Response IP 104.26.14.165
Found Yes
Hash c55eeee42ecb73e9a11785f54447876961a1de463839da0f14c900f4520cc1ba
SimHash 60a55c40c741

Groups

*

Rule Path
Disallow /wp/wp-admin/
Disallow /wp-admin/
Disallow /wp/wp-login.php
Disallow /admin
Disallow */?s=*
Disallow *s%3D*
Disallow *?share=*
Disallow /search/*
Disallow /search?q=*
Allow /

gptbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.beano.com/sitemap.xml
Back to top

Comments

  • Prevent all crawlers from accessing specific parts of the site
  • Allow all crawlers to access everything else
  • Disallow OpenAI crawler
  • Sitemap URL
Back to top