Log In Sign Up

biteoffox.com
robots.txt

Robots Exclusion Standard data for biteoffox.com

Resource Scan

Scan Details

Site Domain biteoffox.com
Base Domain biteoffox.com
Scan Status Ok
Last Scan2025-08-27T23:44:32+00:00
Next Scan 2025-09-03T23:44:32+00:00

Last Scan

Scanned2025-08-27T23:44:32+00:00
URL https://biteoffox.com/robots.txt
Redirect https://www.biteoffox.com/robots.txt
Redirect Domain www.biteoffox.com
Redirect Base biteoffox.com
Domain IPs 13.248.160.137, 76.223.34.124
Redirect IPs 13.33.88.112, 13.33.88.126, 13.33.88.88, 13.33.88.94
Response IP 13.33.88.112
Found Yes
Hash f375909904b4ad9493993e6637a05dffd751a4b62c5f1f126b61cf219d293c18
SimHash f81098f0d607

Groups

*

Rule Path
Disallow /admin/
Disallow /v3/
Disallow /recipe_tool/
Disallow /redirect-to-walmart/
Disallow /elux/
Disallow /widget/test/
Disallow /de/

scrapy
petalbot
timpibot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

ahrefssiteaudit

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.biteoffox.com/sitemap.xml
Back to top

Comments

  • Misbehaving bots
Back to top