comune.crotone.it
robots.txt
Robots Exclusion Standard data for comune.crotone.it
Resource Scan
Scan Details
| Site Domain | comune.crotone.it |
| Base Domain | comune.crotone.it |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-05-16T09:41:47+00:00 |
| Next Scan | 2025-06-15T09:41:47+00:00 |
Last Successful Scan
| Scanned | 2025-03-25T09:40:54+00:00 |
| URL | https://comune.crotone.it/robots.txt |
| Redirect | https://www.comune.crotone.it/robots.txt |
| Redirect Domain | www.comune.crotone.it |
| Redirect Base | comune.crotone.it |
| Domain IPs | 34.248.190.134 |
| Redirect IPs | 18.66.195.124, 18.66.195.38, 18.66.195.8, 18.66.195.89, 2600:9000:2364:3200:1b:3671:4500:93a1, 2600:9000:2364:4400:1b:3671:4500:93a1, 2600:9000:2364:5c00:1b:3671:4500:93a1, 2600:9000:2364:6400:1b:3671:4500:93a1, 2600:9000:2364:b200:1b:3671:4500:93a1, 2600:9000:2364:c600:1b:3671:4500:93a1, 2600:9000:2364:de00:1b:3671:4500:93a1, 2600:9000:2364:fe00:1b:3671:4500:93a1 |
| Response IP | 52.85.49.26 |
| Found | Yes |
| Hash | c8d63def7482193f0d9d01fbe39c06a16a2465f93c14afbe5353cc76fbbbe284 |
| SimHash | 8410da22a3d8 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /flex/tmp/ |
| Disallow | /flex/Logs/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |