femalefirst.co.uk
robots.txt
Robots Exclusion Standard data for femalefirst.co.uk
Resource Scan
Scan Details
| Site Domain | femalefirst.co.uk |
| Base Domain | femalefirst.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-06-15T00:50:50+00:00 |
| Next Scan | 2024-06-22T00:50:50+00:00 |
Last Scan
| Scanned | 2024-06-15T00:50:50+00:00 |
| URL | https://femalefirst.co.uk/robots.txt |
| Redirect | https://www.femalefirst.co.uk/robots.txt |
| Redirect Domain | www.femalefirst.co.uk |
| Redirect Base | femalefirst.co.uk |
| Domain IPs | 104.21.235.167, 104.21.235.168 |
| Redirect IPs | 104.21.235.167, 104.21.235.168 |
| Response IP | 104.21.235.168 |
| Found | Yes |
| Hash | dbee0370564b9d677a1b1fb9f245474835d51a2200ab5e9ed30697cb572ab068 |
| SimHash | b30d4b1ddd12 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /send-to-friend.php |
| Disallow | /send-to-friend/ |
| Disallow | /vote/ |
| Disallow | /comment/add/ |
| Disallow | /catalog/search/ |
| Disallow | /catalog/bag/ |
| Disallow | /catalog/log-in/ |
| Disallow | /catalog/check-out/ |
| Disallow | /facebook/canvas/ |
| Disallow | /board/posting.php |
| Disallow | /auth/ |
| Disallow | /dating/ajax/ |
| Disallow | /newsletter/confirm/ |
| Disallow | /feeds/image-rotator/ |
| Disallow | /~tracker/ |
| Disallow | /heading/ |
| Disallow | /~status/ |
| Disallow | /sex/ |
| Disallow | /lovehoney/ |
| Disallow | /sex-tips/ |
| Disallow | /sexual-advice/ |
| Disallow | /oral-sex/ |
| Disallow | /anal-sex/ |
| Disallow | /sex-life/ |
| Disallow | /porn/ |
| Disallow | /daddy-porn/ |
| Disallow | /penis/ |
| Disallow | /penis-size/ |
| Disallow | /sex-toys/ |
| Disallow | /fam-cms/ |
| Disallow | /auth/sign-in |
| Disallow | /auth/done |
| Disallow | /8350602/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.femalefirst.co.uk/sitemap.xml |
Warnings
- 1 invalid line.
Comments