curiofirenze.com
robots.txt

Robots Exclusion Standard data for curiofirenze.com

Resource Scan

Scan Details

Site Domain curiofirenze.com
Base Domain curiofirenze.com
Scan Status Ok
Last Scan2024-05-18T12:48:13+00:00
Next Scan 2024-06-17T12:48:13+00:00

Last Scan

Scanned2024-05-18T12:48:13+00:00
URL https://curiofirenze.com/robots.txt
Domain IPs 104.26.0.60, 104.26.1.60, 172.67.69.153, 2606:4700:20::681a:13c, 2606:4700:20::681a:3c, 2606:4700:20::ac43:4599
Response IP 104.26.1.60
Found Yes
Hash 311d1d0ae81cb77043d01a8fe7785fbd3e42a168387e23c2cded80e3883e502d
SimHash c4fef49cef98

Groups

*

Rule Path
Disallow /admin/
Disallow /myasg/
Disallow /include/asta-tempo/
Disallow /it/controller.asp
Disallow /uk/controller.asp
Disallow /it/controller-v2.asp
Disallow /uk/controller-v2.asp
Disallow /it/my-panel/*
Disallow /uk/my-panel/*
Disallow /it/my-panel/make-bid.asp
Disallow /uk/my-panel/make-bid.asp
Disallow /it/my-panel/pref-lot.asp
Disallow /uk/my-panel/pref-lot.asp
Disallow /uk/my-panel/condition-report.asp
Disallow /it/my-panel/condition-report.asp
Disallow /it/contatti/richiedi-informazioni.asp*?*
Disallow /uk/contacts/request-information.asp*?*
Disallow /it/contatti/richiedi-informazioni.asp?req=*
Disallow /uk/contacts/request-information.asp?req=*
Disallow /include/comuni*
Disallow /thumb/

ia_archiver

Product Comment
ia_archiver web.archive.org
Rule Path
Disallow /

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path
Disallow /

dataforseobot

Product Comment
dataforseobot DataForSeoBot user agent
Rule Path
Disallow /

petalbot

Product Comment
petalbot PetalBot user agent
Rule Path
Disallow /

ahrefsbot

Product Comment
ahrefsbot AhrefsBot user agent
Rule Path
Disallow /

mj12bot

Product Comment
mj12bot MJ12bot user agent
Rule Path
Disallow /

megaindex.ru

Product Comment
megaindex.ru MegaIndex.ru user agent
Rule Path
Disallow /

serpstatbot

Product Comment
serpstatbot serpstatbot user agent
Rule Path
Disallow /

barkrowler

Product Comment
barkrowler Barkrowler user agent
Rule Path
Disallow /

seokicks

Product Comment
seokicks SEOkicks user agent
Rule Path
Disallow /

rainbot

Product Comment
rainbot RainBot user agent
Rule Path
Disallow /

test-bot

Product Comment
test-bot test-bot user agent
Rule Path
Disallow /

my-tiny-bot

Product Comment
my-tiny-bot my-tiny-bot user agent
Rule Path
Disallow /

fidget-spinner-bot

Product Comment
fidget-spinner-bot fidget-spinner-bot user agent
Rule Path
Disallow /