mumsnet.com
robots.txt
Robots Exclusion Standard data for mumsnet.com
Resource Scan
Scan Details
| Site Domain | mumsnet.com |
| Base Domain | mumsnet.com |
| Scan Status | Ok |
| Last Scan | 2024-06-21T17:48:27+00:00 |
| Next Scan | 2024-06-28T17:48:27+00:00 |
Last Scan
| Scanned | 2024-06-21T17:48:27+00:00 |
| URL | https://mumsnet.com/robots.txt |
| Redirect | https://www.mumsnet.com/robots.txt |
| Redirect Domain | www.mumsnet.com |
| Redirect Base | mumsnet.com |
| Domain IPs | 104.22.40.105, 104.22.41.105, 172.67.7.150, 2606:4700:10::6816:2869, 2606:4700:10::6816:2969, 2606:4700:10::ac43:796 |
| Redirect IPs | 104.22.40.105, 104.22.41.105, 172.67.7.150, 2606:4700:10::6816:2869, 2606:4700:10::6816:2969, 2606:4700:10::ac43:796 |
| Response IP | 104.22.40.105 |
| Found | Yes |
| Hash | f8a51c136adb628935da0983b2fc8262c17438f9911074d93e120add5171b358 |
| SimHash | 0934595b65f2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /signin?* |
| Disallow | /register-user?* |
| Disallow | /ReportTalkPost* |
| Disallow | /version-choice* |
| Disallow | /pm* |
| Disallow | /session/ |
| Disallow | /login-needed/ |
| Disallow | /ajax/* |
| Disallow | /s/A |
| Disallow | /member/ |
| Disallow | /emo/ |
| Disallow | /te/ |
| Disallow | /api/ |
| Disallow | /system/1/ |
| Disallow | /discus/ |
| Disallow | /pies/ |
| Disallow | /talk-pictures/ |
| Disallow | /food/recipe/add-recipe |
| Disallow | /service/searchlambda/all-search-results* |
| Disallow | /SearchArch?* |
| Disallow | |
| Disallow | /utility/* |
| Allow | /.well-known/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.mumsnet.com/sitemap/cms/content |
| sitemap | https://www.mumsnet.com/sitemap/cms/news |
| sitemap | https://www.mumsnet.com/sitemap/talk |
Warnings
- 2 invalid lines.
Comments