Log In Sign Up

boxrec.com
robots.txt

Robots Exclusion Standard data for boxrec.com

Resource Scan

Scan Details

Site Domain boxrec.com
Base Domain boxrec.com
Scan Status Ok
Last Scan2024-05-06T22:01:42+00:00
Next Scan 2024-05-13T22:01:42+00:00

Last Scan

Scanned2024-05-06T22:01:42+00:00
URL https://boxrec.com/robots.txt
Domain IPs 104.22.76.229, 104.22.77.229, 172.67.27.140, 2606:4700:10::6816:4ce5, 2606:4700:10::6816:4de5, 2606:4700:10::ac43:1b8c
Response IP 104.22.76.229
Found Yes
Hash f926eb4f6d7abc026529a25dd37a6b85a6743e58da9526df20186e4e6377483a
SimHash 0cfc0c16c7f5

Groups

*
adsbot-google

Rule Path
Disallow /v6branch/
Disallow /v3/
Disallow /v51static/
Disallow /hu/
Disallow */watch/
Disallow /hugman/
Disallow /media/
Disallow /list_bouts.php
Disallow /show_display.php
Disallow /schedule.php
Disallow /search.php
Disallow /ratings.php
Disallow /title_search.php
Back to top

Comments

  • Block all bots, (AdsBot-Google needs to be explicitly specified) from accessing old areas of the site
Back to top