anuario-horario.es
robots.txt

Robots Exclusion Standard data for anuario-horario.es

Resource Scan

Scan Details

Site Domain anuario-horario.es
Base Domain anuario-horario.es
Scan Status Ok
Last Scan2024-11-12T23:58:24+00:00
Next Scan 2024-11-19T23:58:24+00:00

Last Scan

Scanned2024-11-12T23:58:24+00:00
URL https://anuario-horario.es/robots.txt
Redirect https://www.anuario-horario.es/robots.txt
Redirect Domain www.anuario-horario.es
Redirect Base anuario-horario.es
Domain IPs 195.154.31.74
Redirect IPs 195.154.31.74
Response IP 195.154.31.74
Found Yes
Hash 07e3e27332366593b8f0beed679c104ab91f7cd1e619eb65d63edaec1d6a5655
SimHash c81cc182f1e3

Groups

*

Rule Path
Disallow /vendor
Disallow /abc
Disallow /recherche/lieux?type=&ville=
Disallow /index.php/
Disallow /action/
Disallow /liste-ville*
Disallow /liste-type*
Disallow /liste/proximiter*

ahrefsbot
backlinkcrawler
bdbrandprotect
bpimagewalker
ezooms
findlinks
gigabot
httrack
httrack 3.0
ia_archiver
linkwalker
mj12bot/v1.4.3
mj12bot
net vampire
python-urllib
rogerbot
sogou web spider
sosospider
spbot
updownerbot
semalt.com
robothumb.com
trustpilot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.anuario-horario.es/xml/es.xml
sitemap https://www.anuario-horario.es/xml/es2.xml
sitemap https://www.anuario-horario.es/xml/es3.xml

Comments

  • Crawlers

Warnings

  • 1 invalid line.