dansmediadigest.co.uk
robots.txt
Robots Exclusion Standard data for dansmediadigest.co.uk
Resource Scan
Scan Details
| Site Domain | dansmediadigest.co.uk |
| Base Domain | dansmediadigest.co.uk |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2025-10-21T15:53:46+00:00 |
| Next Scan | 2026-01-19T15:53:46+00:00 |
Last Successful Scan
| Scanned | 2025-06-16T11:14:12+00:00 |
| URL | https://dansmediadigest.co.uk/robots.txt |
| Redirect | https://dansmediadigest.co.uk/robots.txt?gi=936b5dadd592 |
| Domain IPs | 162.159.152.4, 162.159.153.4 |
| Response IP | 162.159.152.4 |
| Found | Yes |
| Hash | bf687f39419f658684f0dcb0d455132758e7951a58bf742f950c1de3f66dfedd |
| SimHash | 6b10fb44c772 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /m/ |
| Disallow | /me/ |
| Disallow | /%40me$ |
| Disallow | /%40me/ |
| Disallow | /*/edit$ |
| Disallow | /*/*/edit$ |
| Disallow | /media/ |
| Disallow | /p/*/share |
| Disallow | /r/ |
| Disallow | /trending |
| Disallow | /search?q$ |
| Disallow | /search?q= |
| Disallow | /*/search?q= |
| Disallow | /*/search/*?q= |
| Disallow | /*/*source%3D |
| Allow | /_/ |
| Allow | /_/api/users/*/meta |
| Allow | /_/api/users/*/profile/stream |
| Allow | /_/api/posts/*/responses |
| Allow | /_/api/posts/*/responsesStream |
| Allow | /_/api/posts/*/related |
Other Records
| Field | Value |
|---|---|
| sitemap | https://dansmediadigest.co.uk/sitemap/sitemap.xml |