annuario-orari.it
robots.txt

Robots Exclusion Standard data for annuario-orari.it

Resource Scan

Scan Details

Site Domain annuario-orari.it
Base Domain annuario-orari.it
Scan Status Ok
Last Scan2024-11-12T23:47:29+00:00
Next Scan 2024-11-19T23:47:29+00:00

Last Scan

Scanned2024-11-12T23:47:29+00:00
URL https://annuario-orari.it/robots.txt
Redirect https://www.annuario-orari.it/robots.txt
Redirect Domain www.annuario-orari.it
Redirect Base annuario-orari.it
Domain IPs 195.154.31.74
Redirect IPs 195.154.31.74
Response IP 195.154.31.74
Found Yes
Hash 2ebe69c73bd377ca684abff5d704ad2a4e8f211e0599e7fefeff0536c309f4cf
SimHash d85c80c2f1c3

Groups

*

Rule Path
Disallow /vendor
Disallow /abc
Disallow /recherche/lieux?type=&ville=
Disallow /index.php/
Disallow /action/
Disallow /liste-ville*
Disallow /liste-type*
Disallow /liste/proximiter*

ahrefsbot
backlinkcrawler
bdbrandprotect
bpimagewalker
ezooms
findlinks
gigabot
httrack
httrack 3.0
ia_archiver
linkwalker
mj12bot/v1.4.3
mj12bot
net vampire
python-urllib
rogerbot
sogou web spider
sosospider
spbot
updownerbot
semalt.com
robothumb.com
trustpilot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.annuario-orari.it/xml/it.xml
sitemap https://www.annuario-orari.it/xml/it2.xml
sitemap https://www.annuario-orari.it/xml/it3.xml

Comments

  • Crawlers

Warnings

  • 1 invalid line.