crystalroof.co.uk
robots.txt
Robots Exclusion Standard data for crystalroof.co.uk
Resource Scan
Scan Details
| Site Domain | crystalroof.co.uk |
| Base Domain | crystalroof.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-06-10T02:55:16+00:00 |
| Next Scan | 2024-06-17T02:55:16+00:00 |
Last Scan
| Scanned | 2024-06-10T02:55:16+00:00 |
| URL | https://crystalroof.co.uk/robots.txt |
| Domain IPs | 13.35.18.115, 13.35.18.120, 13.35.18.4, 13.35.18.45, 2600:9000:20c7:1600:0:65ba:3bc0:93a1, 2600:9000:20c7:2800:0:65ba:3bc0:93a1, 2600:9000:20c7:4400:0:65ba:3bc0:93a1, 2600:9000:20c7:5200:0:65ba:3bc0:93a1, 2600:9000:20c7:6600:0:65ba:3bc0:93a1, 2600:9000:20c7:9600:0:65ba:3bc0:93a1, 2600:9000:20c7:b000:0:65ba:3bc0:93a1, 2600:9000:20c7:f000:0:65ba:3bc0:93a1 |
| Response IP | 13.35.18.115 |
| Found | Yes |
| Hash | d36437ea565e60efeb2e30c682a9b50f3ad906a749c533ae4033e1b91ff18ac8 |
| SimHash | 20344c008693 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /customer-api/ |
| Disallow | /data-api/ |
| Disallow | /layers/ |
| Disallow | /report/postcode/*/affluence |
Other Records
| Field | Value |
|---|---|
| sitemap | https://crystalroof.co.uk/sitemap.xml |