cgpplus.co.uk
robots.txt
Robots Exclusion Standard data for cgpplus.co.uk
Resource Scan
Scan Details
Site Domain | cgpplus.co.uk |
Base Domain | cgpplus.co.uk |
Scan Status | Ok |
Last Scan | 2024-09-24T16:58:57+00:00 |
Next Scan | 2024-10-24T16:58:57+00:00 |
Last Scan
Scanned | 2024-09-24T16:58:57+00:00 |
URL | https://cgpplus.co.uk/robots.txt |
Redirect | https://www.cgpplus.co.uk/robots.txt |
Redirect Domain | www.cgpplus.co.uk |
Redirect Base | cgpplus.co.uk |
Domain IPs | 3.251.18.84 |
Redirect IPs | 13.248.175.244, 76.223.54.145 |
Response IP | 13.248.175.244 |
Found | Yes |
Hash | 219e045427abbd4295e751698e6c19101bcb66c133c6d8a4f06e3891bca5ecfc |
SimHash | f345277f2baf |
Groups
*
Rule | Path |
---|---|
Disallow | /admin/ |
Disallow | /csr/ |
Disallow | /modules/ |
Disallow | /app_data/ |
Disallow | /*?cookiesViewed=true |
Disallow | /*?c= |
Disallow | /search-results* |
Other Records
Field | Value |
---|---|
crawl-delay | 6 |
Other Records
Field | Value |
---|---|
sitemap | https://www.cgpbooks.co.uk/google-sitemap |
Warnings
- 8 invalid lines.
Comments