bglinsurance.co.uk
robots.txt
Robots Exclusion Standard data for bglinsurance.co.uk
Resource Scan
Scan Details
| Site Domain | bglinsurance.co.uk |
| Base Domain | bglinsurance.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-11-06T02:16:14+00:00 |
| Next Scan | 2024-12-06T02:16:14+00:00 |
Last Scan
| Scanned | 2024-11-06T02:16:14+00:00 |
| URL | https://bglinsurance.co.uk/robots.txt |
| Redirect | https://www.bglinsurance.co.uk/robots.txt |
| Redirect Domain | www.bglinsurance.co.uk |
| Redirect Base | bglinsurance.co.uk |
| Domain IPs | 23.215.7.12, 23.215.7.25, 2600:1413:b000:1b::17d7:70c, 2600:1413:b000:1b::17d7:719 |
| Redirect IPs | 23.215.7.12, 23.215.7.25, 2600:1413:b000:1b::17d7:70c, 2600:1413:b000:1b::17d7:719 |
| Response IP | 96.17.180.49 |
| Found | Yes |
| Hash | 80bac33008415c10333969597fd8137ce34e39f06753d4ae72adba08efbb1d9c |
| SimHash | 7c5d4826c702 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /flare/ |
| Disallow | /javascript/ |
| Disallow | /movies/ |
| Disallow | /include/ |
| Disallow | /docs/ |
| Disallow | /logs/ |
| Disallow | /home_page_popups/ |
| Disallow | /pages/ |
| Disallow | /partners/ |
| Disallow | /row2/ |
| Disallow | /scripts/ |
| Disallow | /site/ |
| Disallow | /thumbs/ |
| Disallow | /row2/maps/ |
| Disallow | /row2/flash/ |
| Disallow | /row2/email/ |
| Disallow | /site2/ |
| Disallow | /us2/ |
| Disallow | /eu2/ |
| Disallow | /au2/ |
| Disallow | /gw/ |
| Disallow | /foo/ |
| Disallow | /newblog/ |
| Disallow | /newnews/ |
| Disallow | /CVS/ |
| Disallow | */document-retention-policy/ |
| Disallow | */usage-policy/ |
| Disallow | */terms-and-conditions/ |
| Disallow | */privacy-policy/ |
| Disallow | /agents-area/ |
| Disallow | /partners/ |
| Disallow | /operators-log-in/ |
| Disallow | /static-pages/white-label |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.markerstudygroup.com/sitemap.xml |