distriauto.fr
robots.txt

Robots Exclusion Standard data for distriauto.fr

Resource Scan

Scan Details

Site Domain distriauto.fr
Base Domain distriauto.fr
Scan Status Ok
Last Scan2024-10-05T04:32:40+00:00
Next Scan 2024-11-04T04:32:40+00:00

Last Scan

Scanned2024-10-05T04:32:40+00:00
URL https://distriauto.fr/robots.txt
Redirect https://www.distriauto.fr/robots.txt
Redirect Domain www.distriauto.fr
Redirect Base distriauto.fr
Domain IPs 162.19.190.255
Redirect IPs 162.19.190.255
Response IP 162.19.190.255
Found Yes
Hash c6eae0e64fdf4234fdaa073d0431bbbd75b645250ff86ff67e1758e4c52db3ed
SimHash a74c5e128420

Groups

*

Rule Path
Disallow /*.ppt$
Disallow /*.csv$
Disallow /*.xls$
Disallow /*.xlsx$
Disallow /*.zip$
Disallow /*.rar$
Disallow /cart*
Disallow /account*
Disallow /file*
Disallow /regularization*
Disallow /contact/succes
Disallow /pieces-auto/recherche
Disallow /accessoires/recherche
Disallow /ajax/*
Disallow /spare-parts/document/*

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

vbot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

awariobot
awariorssbot
awariosmartbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.distriauto.fr/sitemap/distriauto.fr/sitemap.xml