Log In Sign Up

whichcar.com.au
robots.txt

Robots Exclusion Standard data for whichcar.com.au

Resource Scan

Scan Details

Site Domain whichcar.com.au
Base Domain whichcar.com.au
Scan Status Ok
Last Scan2024-11-09T06:43:09+00:00
Next Scan 2024-11-16T06:43:09+00:00

Last Scan

Scanned2024-11-09T06:43:09+00:00
URL https://whichcar.com.au/robots.txt
Redirect https://www.whichcar.com.au/robots.txt
Redirect Domain www.whichcar.com.au
Redirect Base whichcar.com.au
Domain IPs 2600:9000:271a:4a00:7:8fc7:5380:93a1, 2600:9000:271a:5600:7:8fc7:5380:93a1, 2600:9000:271a:7200:7:8fc7:5380:93a1, 2600:9000:271a:a800:7:8fc7:5380:93a1, 2600:9000:271a:b400:7:8fc7:5380:93a1, 2600:9000:271a:c200:7:8fc7:5380:93a1, 2600:9000:271a:c400:7:8fc7:5380:93a1, 2600:9000:271a:e000:7:8fc7:5380:93a1, 3.165.82.100, 3.165.82.109, 3.165.82.8, 3.165.82.93
Redirect IPs 2600:9000:271a:1e00:7:8fc7:5380:93a1, 2600:9000:271a:2800:7:8fc7:5380:93a1, 2600:9000:271a:ac00:7:8fc7:5380:93a1, 2600:9000:271a:cc00:7:8fc7:5380:93a1, 2600:9000:271a:d600:7:8fc7:5380:93a1, 2600:9000:271a:e000:7:8fc7:5380:93a1, 2600:9000:271a:ee00:7:8fc7:5380:93a1, 2600:9000:271a:f800:7:8fc7:5380:93a1, 3.165.82.100, 3.165.82.109, 3.165.82.8, 3.165.82.93
Response IP 3.165.82.100
Found Yes
Hash bfd68e518e830a0c9a228dff81da738ad7b31d1f919098e69b271de967f0b0b4
SimHash 671fd860c3d1

Groups

*

Rule Path
Disallow /_*
Disallow /search-post/*

Other Records

Field Value
crawl-delay 5

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

riddler

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.whichcar.com.au/sitemap/sitemap-index.xml
sitemap https://www.whichcar.com.au/googlenews
Back to top