geektown.co.uk
robots.txt
Robots Exclusion Standard data for geektown.co.uk
Resource Scan
Scan Details
| Site Domain | geektown.co.uk |
| Base Domain | geektown.co.uk |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-10-19T17:02:28+00:00 |
| Next Scan | 2025-01-17T17:02:28+00:00 |
Last Successful Scan
| Scanned | 2022-12-28T03:01:00+00:00 |
| URL | https://geektown.co.uk/robots.txt |
| Redirect | https://www.geektown.co.uk/robots.txt |
| Redirect Domain | www.geektown.co.uk |
| Redirect Base | geektown.co.uk |
| Domain IPs | 91.197.230.20 |
| Redirect IPs | 104.26.8.48, 104.26.9.48, 172.67.72.67, 2606:4700:20::681a:830, 2606:4700:20::681a:930, 2606:4700:20::ac43:4843 |
| Response IP | 104.26.8.48 |
| Found | Yes |
| Hash | 42d2ab02db3aee7bbdda0f524fcc400b93ca44b3d1cc9c7e575881a354969467 |
| SimHash | 6375955446c7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin |
| Disallow | /wp-admin |
| Disallow | /wp-includes |
| Disallow | /wp-content/plugins |
| Disallow | /wp-content/cache |
| Disallow | /wp-content/themes |
| Disallow | /category |
| Disallow | /tag |
| Disallow | /author |
| Disallow | /trackback |
| Disallow | /*trackback |
| Disallow | /*trackback* |
| Disallow | /*/trackback |
| Disallow | /*?* |
| Disallow | /*.html/$ |
| Disallow | /*feed* |
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.geektown.co.uk/post-sitemap1.xml |
| sitemap | http://www.geektown.co.uk/post-sitemap2.xml |
| sitemap | http://www.geektown.co.uk/page-sitemap.xml |
| sitemap | http://www.geektown.co.uk/tvairdates-sitemap.xml |
| sitemap | http://www.geektown.co.uk/category-sitemap.xml |
| sitemap | http://www.geektown.co.uk/author-sitemap.xml |
Comments