mercedesbenzofdenver.com
robots.txt

Robots Exclusion Standard data for mercedesbenzofdenver.com

Resource Scan

Scan Details

Site Domain mercedesbenzofdenver.com
Base Domain mercedesbenzofdenver.com
Scan Status Ok
Last Scan2024-10-19T10:23:07+00:00
Next Scan 2024-11-18T10:23:07+00:00

Last Scan

Scanned2024-10-19T10:23:07+00:00
URL https://mercedesbenzofdenver.com/robots.txt
Redirect https://www.mercedesbenzofdenver.com/robots.txt
Redirect Domain www.mercedesbenzofdenver.com
Redirect Base mercedesbenzofdenver.com
Domain IPs 23.32.29.107, 23.32.29.99, 2600:1413:1::1720:1d63, 2600:1413:1::1720:1d6b
Redirect IPs 23.215.7.10, 23.215.7.20, 2600:1413:b000:1d::17d1:2e8f, 2600:1413:b000:1d::17d1:2e9b
Response IP 23.32.29.99
Found Yes
Hash f4da69557f7f3cb252e7e4ae1bb22c3aa6cb9720678816580c1ad20d7f7fc1f7
SimHash 4d7efb3fc0a0

Groups

imagewalker

Rule Path
Disallow /

gsa-crawler

Rule Path
Disallow /

omniexplorer_bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

trovitbot

Rule Path
Disallow /

realspider

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow /api/pse/
Disallow /api/legacy/pse/
Disallow /external-catalog-services/
Disallow /tcd/
Disallow /esntial.htm
Disallow /microsite/
Disallow /*ajax.htm$
Disallow /*fragment.htm
Disallow /image/viewer.htm
Disallow /apis/mycars/
Disallow /mycars/
Disallow /reserve-it-now/form.htm
Disallow /contact-form.htm
Disallow /specials/trims.htm
Disallow /inventory-detail/print.htm
Disallow /blog/tags/*

adsbot-google-mobile

Rule Path
Disallow /api/pse/
Disallow /api/legacy/pse/
Disallow /external-catalog-services/
Disallow /tcd/
Disallow /esntial.htm
Disallow /microsite/
Disallow /*ajax.htm$
Disallow /*fragment.htm
Disallow /image/viewer.htm
Disallow /apis/mycars/
Disallow /mycars/
Disallow /reserve-it-now/form.htm
Disallow /contact-form.htm
Disallow /specials/trims.htm
Disallow /inventory-detail/print.htm
Disallow /blog/tags/*

mj12bot

Rule Path
Disallow /api/pse/
Disallow /api/legacy/pse/
Disallow /external-catalog-services/
Disallow /tcd/
Disallow /esntial.htm
Disallow /microsite/
Disallow /*ajax.htm$
Disallow /*fragment.htm
Disallow /image/viewer.htm
Disallow /apis/mycars/
Disallow /mycars/
Disallow /reserve-it-now/form.htm
Disallow /contact-form.htm
Disallow /specials/trims.htm
Disallow /inventory-detail/print.htm
Disallow /blog/tags/*

Other Records

Field Value
crawl-delay 20

slurp

Rule Path
Disallow /api/pse/
Disallow /api/legacy/pse/
Disallow /external-catalog-services/
Disallow /tcd/
Disallow /esntial.htm
Disallow /microsite/
Disallow /*ajax.htm$
Disallow /*fragment.htm
Disallow /image/viewer.htm
Disallow /apis/mycars/
Disallow /mycars/
Disallow /reserve-it-now/form.htm
Disallow /contact-form.htm
Disallow /specials/trims.htm
Disallow /inventory-detail/print.htm
Disallow /blog/tags/*

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /api/pse/
Disallow /api/legacy/pse/
Disallow /external-catalog-services/
Disallow /tcd/
Disallow /esntial.htm
Disallow /microsite/
Disallow /*ajax.htm$
Disallow /*fragment.htm
Disallow /image/viewer.htm
Disallow /apis/mycars/
Disallow /mycars/
Disallow /reserve-it-now/form.htm
Disallow /contact-form.htm
Disallow /specials/trims.htm
Disallow /inventory-detail/print.htm
Disallow /blog/tags/*

Other Records

Field Value
sitemap https://www.mercedesbenzofdenver.com/sitemap.xml
sitemap https://www.mercedesbenzofdenver.com/videositemap.xml