wineaustralia.com
robots.txt

Robots Exclusion Standard data for wineaustralia.com

Resource Scan

Scan Details

Site Domain wineaustralia.com
Base Domain wineaustralia.com
Scan Status Ok
Last Scan2024-09-28T20:50:58+00:00
Next Scan 2024-10-28T20:50:58+00:00

Last Scan

Scanned2024-09-28T20:50:58+00:00
URL https://wineaustralia.com/robots.txt
Redirect https://www.wineaustralia.com/robots.txt
Redirect Domain www.wineaustralia.com
Redirect Base wineaustralia.com
Domain IPs 20.37.196.201
Redirect IPs 13.107.246.59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash 912930e851fce496ccbaf701a2a2fb1265c1af619e5d15c7ec361f954617eed1
SimHash 4954cb0164f0

Groups

*

Rule Path
Allow /

trendkite-akashic-crawler

Rule Path
Disallow /

*

Rule Path
Disallow /getmedia/*

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

Comments

  • Block trendkite-akashic-crawler
  • Block all media files
  • Allow google all