origin.getprice.com.au
robots.txt

Robots Exclusion Standard data for origin.getprice.com.au

Resource Scan

Scan Details

Site Domain origin.getprice.com.au
Base Domain getprice.com.au
Scan Status Ok
Last Scan2024-10-27T13:08:04+00:00
Next Scan 2024-11-26T13:08:04+00:00

Last Scan

Scanned2024-10-27T13:08:04+00:00
URL https://origin.getprice.com.au/robots.txt
Domain IPs 40.126.230.185
Response IP 40.126.230.185
Found Yes
Hash 78247688b777875b6c6b9353351e58ee3eb5a6dfa930c7dad0211b474d4c92d6
SimHash 0555db00f791

Groups

*

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow /

geedoproductsearch

Rule Path
Disallow /

brandverity

Rule Path
Disallow /

Comments

  • Block all crawlers except AdsBot (AdsBot crawlers must be named explicitly)
  • Block AdsBot explicitly