santcugat.cat
robots.txt
Robots Exclusion Standard data for santcugat.cat
Resource Scan
Scan Details
Site Domain | santcugat.cat |
Base Domain | santcugat.cat |
Scan Status | Failed |
Failure Reason | Scan timed out. |
Last Scan | 2024-09-08T18:17:21+00:00 |
Next Scan | 2024-12-07T18:17:21+00:00 |
Last Successful Scan
Scanned | 2024-04-19T14:00:28+00:00 |
URL | https://santcugat.cat/robots.txt |
Domain IPs | 46.27.195.18 |
Response IP | 46.27.195.18 |
Found | Yes |
Hash | eebdba2f85fff1ebe81162235e87c4c563d03da16c7646602ee8cf4dc39f6cff |
SimHash | 12584440917a |
Groups
*
Rule | Path |
---|---|
Disallow | /ajax/ |
Disallow | /common/ |
Disallow | /downloads/ |
Disallow | /handlers/ |
Disallow | /images/ |
Disallow | /includes/ |
*
No rules defined. All paths allowed.
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
Warnings
- 2 invalid lines.
Comments