Log In Sign Up

sanebox.com
robots.txt

Robots Exclusion Standard data for sanebox.com

Resource Scan

Scan Details

Site Domain sanebox.com
Base Domain sanebox.com
Scan Status Ok
Last Scan2025-12-05T04:14:14+00:00
Next Scan 2025-12-19T04:14:14+00:00

Last Scan

Scanned2025-12-05T04:14:14+00:00
URL https://sanebox.com/robots.txt
Redirect https://www.sanebox.com/robots.txt
Redirect Domain www.sanebox.com
Redirect Base sanebox.com
Domain IPs 104.26.10.163, 104.26.11.163, 172.67.73.132, 2606:4700:20::681a:aa3, 2606:4700:20::681a:ba3, 2606:4700:20::ac43:4984
Redirect IPs 104.26.10.163, 104.26.11.163, 172.67.73.132, 2606:4700:20::681a:aa3, 2606:4700:20::681a:ba3, 2606:4700:20::ac43:4984
Response IP 104.26.11.163
Found Yes
Hash 356d53be3f37d4f90bc785fc5f7f8ce362d279de612fcb6ead09a215d9f17909
SimHash eaae0e8569f0

Groups

bitlybot

Rule Path
Disallow /

*

Rule Path
Disallow /api
Disallow /bp
Disallow /confirm
Disallow /dashboard
Disallow /dig
Disallow /ignore_list
Disallow /reminders
Disallow /train
Disallow /training
Disallow /undo
Disallow /exp

semrushbot

Rule Path
Disallow /help
Back to top

Other Records

Field Value
sitemap http://www.sanebox.com/sitemap.xml
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-Agent: *
  • Disallow: /
Back to top