northlandchevrolet.net
robots.txt

Robots Exclusion Standard data for northlandchevrolet.net

Resource Scan

Scan Details

Site Domain northlandchevrolet.net
Base Domain northlandchevrolet.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-25T19:06:36+00:00
Next Scan 2024-12-24T19:06:36+00:00

Last Successful Scan

Scanned2023-02-11T01:53:23+00:00
URL https://northlandchevrolet.net/robots.txt
Redirect https://www.northlandchevrolet.net/robots.txt
Redirect Domain www.northlandchevrolet.net
Redirect Base northlandchevrolet.net
Domain IPs 216.241.213.55
Redirect IPs 18.155.68.100, 18.155.68.21, 18.155.68.44, 18.155.68.58
Response IP 18.155.68.58
Found Yes
Hash de9392ac09c56a95fe688d021e067808d7609bc82a0e5d2f25563c87da4a2260
SimHash 2b1010a0c6b0

Groups

googlebot
googlebot-mobile
adsbot-google

Rule Path
Disallow /siteMap
Disallow /*.do
Disallow /*.ajax
Disallow /*.js
Disallow /*.css
Disallow /f_*

bingbot
applebot
msnbot
adidxbot
motominerbot
mj12bot
rogerbot
ravencrawler
twitterbot
slurp
duckduckbot
semrushbot
teoma
archive.org_bot

Rule Path
Disallow /siteMap
Disallow /*.do
Disallow /*.ajax
Disallow /*.js
Disallow /*.css
Disallow /f_*

Other Records

Field Value
crawl-delay 20

*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.northlandchevrolet.net/sitemap.xml