Log In Sign Up

gamersunite.com
robots.txt

Robots Exclusion Standard data for gamersunite.com

Resource Scan

Scan Details

Site Domain gamersunite.com
Base Domain gamersunite.com
Scan Status Ok
Last Scan2025-09-28T02:35:34+00:00
Next Scan 2025-10-05T02:35:34+00:00

Last Scan

Scanned2025-09-28T02:35:34+00:00
URL https://gamersunite.com/robots.txt
Redirect https://www.gamersunite.com/robots.txt
Redirect Domain www.gamersunite.com
Redirect Base gamersunite.com
Domain IPs 104.21.11.134, 172.67.166.32, 2606:4700:3034::6815:b86, 2606:4700:3037::ac43:a620
Redirect IPs 104.21.11.134, 172.67.166.32, 2606:4700:3034::6815:b86, 2606:4700:3037::ac43:a620
Response IP 172.67.166.32
Found Yes
Hash aac34370fe477839e4efe1e67526d2177545b8e64d98e9a0b381d65c4227f10f
SimHash a68d0d8d6570

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
Back to top