gty.org
robots.txt

Robots Exclusion Standard data for gty.org

Resource Scan

Scan Details

Site Domain gty.org
Base Domain gty.org
Scan Status Ok
Last Scan2025-12-16T07:58:55+00:00
Next Scan 2026-01-15T07:58:55+00:00

Last Scan

Scanned2025-12-16T07:58:55+00:00
URL https://gty.org/robots.txt
Redirect https://www.gty.org/robots.txt
Redirect Domain www.gty.org
Redirect Base gty.org
Domain IPs 104.18.6.217, 104.18.7.217, 2606:4700::6812:6d9, 2606:4700::6812:7d9
Redirect IPs 104.18.6.217, 104.18.7.217, 2606:4700::6812:6d9, 2606:4700::6812:7d9
Response IP 104.18.6.217
Found Yes
Hash b818dcc41a3c1782aa5e53c6b03cd11014a4fea8aaa96d8825ca0f6fedeb9e4b
SimHash 4555d9047790

Groups

googlebot

Rule Path
Disallow

bingbot

Rule Path
Disallow

adsbot-google

Rule Path
Disallow /

*

Rule Path
Disallow /

Comments

  • Enables Googlebot
  • Allow Bingbot
  • Block Adsbot
  • Example 3: Block all crawlers except AdsBot (AdsBot crawlers must be named explicitly)