astecity.it
robots.txt

Robots Exclusion Standard data for astecity.it

Resource Scan

Scan Details

Site Domain astecity.it
Base Domain astecity.it
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-09-22T01:55:58+00:00
Next Scan 2025-11-21T01:55:58+00:00

Last Successful Scan

Scanned2025-07-25T01:25:09+00:00
URL https://astecity.it/robots.txt
Redirect https://www.astecity.it/robots.txt
Redirect Domain www.astecity.it
Redirect Base astecity.it
Domain IPs 80.211.159.35
Redirect IPs 80.211.159.35
Response IP 80.211.159.35
Found Yes
Hash 2b1708cd0f9d4bd6dc49180f5fc9c7ae473f6779568bd243faf11536563ecc73
SimHash f531c2e68dd3

Groups

*

Rule Path
Disallow /*.axd

*

Rule Path
Disallow /*.thm

*

Rule Path
Disallow /vendite_giudiziarie/aste_giudiziarie.aspx

webmirror

Rule Path
Disallow /

webcopy

Rule Path
Disallow /

winhttrack

Rule Path
Disallow /

website extractor

Rule Path
Disallow /

*

Rule Path
Disallow /focus/

superbot

Rule Path
Disallow /

superbot/2.6

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

black hole

Rule Path
Disallow /

titan

Rule Path
Disallow /

netmechanic

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

disco pump 3.1

Rule Path
Disallow /

netattache

Rule Path
Disallow /

netattache light 1.1

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

copyrightcheck

Rule Path
Disallow /

crescent

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

prowebwalker

Rule Path
Disallow /

cheesebot

Rule Path
Disallow /