xter.io
robots.txt

Robots Exclusion Standard data for xter.io

Resource Scan

Scan Details

Site Domain xter.io
Base Domain xter.io
Scan Status Ok
Last Scan2024-07-01T23:39:32+00:00
Next Scan 2024-07-31T23:39:32+00:00

Last Scan

Scanned2024-07-01T23:39:32+00:00
URL https://xter.io/robots.txt
Domain IPs 3.165.102.14, 3.165.102.19, 3.165.102.33, 3.165.102.41
Response IP 3.165.102.14
Found Yes
Hash cd28d4fc6450c1304943b6f4ae571d685c03504efab8c35501637a9fa3726c39
SimHash 45119b127791

Groups

*

Rule Path
Allow /

googlebot
adsbot-google

Rule Path
Allow /

Comments

  • https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt
  • Block all crawlers except AdsBot (AdsBot crawlers must be named explicitly)
  • Block Googlebot and Adsbot