barstoolgolftime.com
robots.txt

Robots Exclusion Standard data for barstoolgolftime.com

Resource Scan

Scan Details

Site Domain barstoolgolftime.com
Base Domain barstoolgolftime.com
Scan Status Ok
Last Scan2024-11-05T06:32:14+00:00
Next Scan 2024-11-19T06:32:14+00:00

Last Scan

Scanned2024-11-05T06:32:14+00:00
URL https://barstoolgolftime.com/robots.txt
Domain IPs 104.21.10.29, 172.67.189.222, 2606:4700:3035::6815:a1d, 2606:4700:3037::ac43:bdde
Response IP 172.67.189.222
Found Yes
Hash 59682c64a03ab7cc2bd981da05697cb5c4e566adfceb285414075637ad663117
SimHash 445cd0826192

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

googlebot

Rule Path
Disallow

screaming frog

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

Comments

  • https://www.robotstxt.org/robotstxt.html