Log In Sign Up

whichcar.com.au
robots.txt

Robots Exclusion Standard data for whichcar.com.au

Resource Scan

Scan Details

Site Domain whichcar.com.au
Base Domain whichcar.com.au
Scan Status Ok
Last Scan2024-06-26T17:49:28+00:00
Next Scan 2024-07-03T17:49:28+00:00

Last Scan

Scanned2024-06-26T17:49:28+00:00
URL https://whichcar.com.au/robots.txt
Redirect https://www.whichcar.com.au/robots.txt
Redirect Domain www.whichcar.com.au
Redirect Base whichcar.com.au
Domain IPs 13.33.88.101, 13.33.88.124, 13.33.88.42, 13.33.88.55, 2600:9000:2816:1600:7:8fc7:5380:93a1, 2600:9000:2816:6200:7:8fc7:5380:93a1, 2600:9000:2816:7200:7:8fc7:5380:93a1, 2600:9000:2816:9800:7:8fc7:5380:93a1, 2600:9000:2816:d200:7:8fc7:5380:93a1, 2600:9000:2816:e400:7:8fc7:5380:93a1, 2600:9000:2816:ea00:7:8fc7:5380:93a1, 2600:9000:2816:f200:7:8fc7:5380:93a1
Redirect IPs 13.33.88.101, 13.33.88.124, 13.33.88.42, 13.33.88.55, 2600:9000:223b:2c00:7:8fc7:5380:93a1, 2600:9000:223b:3c00:7:8fc7:5380:93a1, 2600:9000:223b:4600:7:8fc7:5380:93a1, 2600:9000:223b:5200:7:8fc7:5380:93a1, 2600:9000:223b:7a00:7:8fc7:5380:93a1, 2600:9000:223b:8400:7:8fc7:5380:93a1, 2600:9000:223b:8600:7:8fc7:5380:93a1, 2600:9000:223b:e800:7:8fc7:5380:93a1
Response IP 13.33.88.42
Found Yes
Hash bfd68e518e830a0c9a228dff81da738ad7b31d1f919098e69b271de967f0b0b4
SimHash 671fd860c3d1

Groups

*

Rule Path
Disallow /_*
Disallow /search-post/*

Other Records

Field Value
crawl-delay 5

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

riddler

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.whichcar.com.au/sitemap/sitemap-index.xml
sitemap https://www.whichcar.com.au/googlenews
Back to top