theboxery.com
robots.txt

Robots Exclusion Standard data for theboxery.com

Resource Scan

Scan Details

Site Domain theboxery.com
Base Domain theboxery.com
Scan Status Ok
Last Scan2025-12-21T17:53:35+00:00
Next Scan 2026-01-20T17:53:35+00:00

Last Scan

Scanned2025-12-21T17:53:35+00:00
URL https://theboxery.com/robots.txt
Redirect https://www.theboxery.com/robots.txt
Redirect Domain www.theboxery.com
Redirect Base theboxery.com
Domain IPs 104.26.6.80, 104.26.7.80, 172.67.73.70, 2606:4700:20::681a:650, 2606:4700:20::681a:750, 2606:4700:20::ac43:4946
Redirect IPs 104.26.6.80, 104.26.7.80, 172.67.73.70, 2606:4700:20::681a:650, 2606:4700:20::681a:750, 2606:4700:20::ac43:4946
Response IP 104.26.7.80
Found Yes
Hash 82efc3e5002ff5a96c07def659f0a17c4f6d243e972b3cefa0b288458bd71152
SimHash 8041da648513

Groups

baiduspider

Rule Path
Disallow /

*

Rule Path
Disallow /old

Other Records

Field Value
sitemap https://www.theboxery.com/sitemap.xml

Comments

  • chinese spider, taking way too many resources, added by M.N. 6/26/2014