Log In Sign Up

austrialisting.com
robots.txt

Robots Exclusion Standard data for austrialisting.com

Resource Scan

Scan Details

Site Domain austrialisting.com
Base Domain austrialisting.com
Scan Status Ok
Last Scan4/20/2025, 1:24:54 PM
Next Scan 4/27/2025, 1:24:54 PM

Last Scan

Scanned4/20/2025, 1:24:54 PM
URL https://austrialisting.com/robots.txt
Redirect http://www.austrialisting.com/robots.txt
Redirect Domain www.austrialisting.com
Redirect Base austrialisting.com
Domain IPs 104.21.37.84, 172.67.206.35, 2606:4700:3031::ac43:ce23, 2606:4700:3037::6815:2554
Redirect IPs 104.21.37.84, 172.67.206.35, 2606:4700:3031::ac43:ce23, 2606:4700:3037::6815:2554
Response IP 172.67.206.35
Found Yes
Hash 34d4fa6e9608ed280b7f634b04e728c7290b6f08253673f0d39cb835a2317c19
SimHash 8c16753766d6

Groups

googlebot

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

bingbot

Rule Path
Disallow

msnbot

Rule Path
Disallow

twitterbot

Rule Path
Disallow

*

Rule Path
Disallow /
Back to top

Comments

  • By accessing this website you agree to Austrialisting.com Terms of Service, available at
  • http://www.austrialisting.com/term_condition
  • If you would like to inquire about crawling austrialisting.com, please contact us at
  • http://www.austrialisting.com/contact_us
  • As always, Asimov's Three Laws are in effect:
  • 1. A robot may not injure a human being or, through inaction, allow a human
  • being to come to harm.
  • 2. A robot must obey orders given it by human beings except where such
  • orders would conflict with the First Law.
  • 3. A robot must protect its own existence as long as such protection does
  • not conflict with the First or Second Law.
Back to top