portal.bps.org.uk
robots.txt
Robots Exclusion Standard data for portal.bps.org.uk
Resource Scan
Scan Details
| Site Domain | portal.bps.org.uk |
| Base Domain | bps.org.uk |
| Scan Status | Ok |
| Last Scan | 2025-12-10T13:45:13+00:00 |
| Next Scan | 2026-01-09T13:45:13+00:00 |
Last Scan
| Scanned | 2025-12-10T13:45:13+00:00 |
| URL | https://portal.bps.org.uk/robots.txt |
| Domain IPs | 104.26.2.23, 104.26.3.23, 172.67.72.207 |
| Response IP | 172.67.72.207 |
| Found | Yes |
| Hash | 1285815f5aab4a7627c1cc57fde48e01afb076c14f7cb171c46eb2d505bd1b4b |
| SimHash | 3b0d1155c5c5 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | /admin/ | - |
| Disallow | /App_Browsers/ | - |
| Disallow | /App_Code/ | - |
| Disallow | /App_Data/ | - |
| Disallow | /App_GlobalResources/ | - |
| Disallow | /bin/ | - |
| Disallow | /Components/ | - |
| Disallow | /Config/ | - |
| Disallow | /contest/ | - |
| Disallow | /controls/ | - |
| Disallow | /DesktopModules/ | - |
| Disallow | /Documentation/ | - |
| Disallow | /HttpModules/ | - |
| Disallow | /images/ | - |
| Disallow | /Install/ | - |
| Disallow | /js/ | - |
| Disallow | /Portals/ | - |
| Disallow | /Providers/ | - |
| Disallow | /Resources/ContentRotator/ | - |
| Disallow | /Resources/ControlPanel/ | - |
| Disallow | /Resources/Dashboard/ | - |
| Disallow | /Resources/FeedBrowser/ | - |
| Disallow | /Resources/OpenForceAd/ | - |
| Disallow | /Resources/Search/ | - |
| Disallow | /Resources/Shared/ | - |
| Disallow | /Resources/SkinWidgets/ | - |
| Disallow | /Resources/TabStrip/ | - |
| Disallow | /Resources/Widgets/ | - |
| Disallow | /Activity-Feed/userId/ | Do not index user profiles |
| Disallow | /*.asmx$ | - |
| Disallow | /DesktopModules/SilverbearCrm/FetchXMLSearch/Data/searchform.asmx | - |
msnbot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /admin/ | - |
| Disallow | /App_Browsers/ | - |
| Disallow | /App_Code/ | - |
| Disallow | /App_Data/ | - |
| Disallow | /App_GlobalResources/ | - |
| Disallow | /bin/ | - |
| Disallow | /Components/ | - |
| Disallow | /Config/ | - |
| Disallow | /contest/ | - |
| Disallow | /controls/ | - |
| Disallow | /DesktopModules/ | - |
| Disallow | /Documentation/ | - |
| Disallow | /HttpModules/ | - |
| Disallow | /images/ | - |
| Disallow | /Install/ | - |
| Disallow | /js/ | - |
| Disallow | /Portals/ | - |
| Disallow | /Providers/ | - |
| Disallow | /Resources/ContentRotator/ | - |
| Disallow | /Resources/ControlPanel/ | - |
| Disallow | /Resources/Dashboard/ | - |
| Disallow | /Resources/FeedBrowser/ | - |
| Disallow | /Resources/OpenForceAd/ | - |
| Disallow | /Resources/Search/ | - |
| Disallow | /Resources/Shared/ | - |
| Disallow | /Resources/SkinWidgets/ | - |
| Disallow | /Resources/TabStrip/ | - |
| Disallow | /Resources/Widgets/ | - |
| Disallow | /Activity-Feed/userId/ | Do not index user profiles |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
slurp
| Rule | Path | Comment |
|---|---|---|
| Disallow | /*/ctl/ | Slurp permits * |
| Disallow | /admin/ | - |
| Disallow | /App_Browsers/ | - |
| Disallow | /App_Code/ | - |
| Disallow | /App_Data/ | - |
| Disallow | /App_GlobalResources/ | - |
| Disallow | /bin/ | - |
| Disallow | /Components/ | - |
| Disallow | /Config/ | - |
| Disallow | /contest/ | - |
| Disallow | /controls/ | - |
| Disallow | /DesktopModules/ | - |
| Disallow | /Documentation/ | - |
| Disallow | /HttpModules/ | - |
| Disallow | /images/ | - |
| Disallow | /Install/ | - |
| Disallow | /js/ | - |
| Disallow | /Portals/ | - |
| Disallow | /Providers/ | - |
| Disallow | /Resources/ContentRotator/ | - |
| Disallow | /Resources/ControlPanel/ | - |
| Disallow | /Resources/Dashboard/ | - |
| Disallow | /Resources/FeedBrowser/ | - |
| Disallow | /Resources/OpenForceAd/ | - |
| Disallow | /Resources/Search/ | - |
| Disallow | /Resources/Shared/ | - |
| Disallow | /Resources/SkinWidgets/ | - |
| Disallow | /Resources/TabStrip/ | - |
| Disallow | /Resources/Widgets/ | - |
| Disallow | /Activity-Feed/userId/ | Do not index user profiles |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
googlebot
| Rule | Path | Comment |
|---|---|---|
| Disallow | /*/ctl/ | Googlebot permits * |
| Disallow | /admin/ | - |
| Disallow | /App_Browsers/ | - |
| Disallow | /App_Code/ | - |
| Disallow | /App_Data/ | - |
| Disallow | /App_GlobalResources/ | - |
| Disallow | /bin/ | - |
| Disallow | /Components/ | - |
| Disallow | /Config/ | - |
| Disallow | /contest/ | - |
| Disallow | /controls/ | - |
| Disallow | /Documentation/ | - |
| Disallow | /HttpModules/ | - |
| Disallow | /Install/ | - |
| Disallow | /Providers/ | - |
| Disallow | /Activity-Feed/userId/ | Do not index user profiles |
| Disallow | /*.asmx$ | - |
| Disallow | /DesktopModules/SilverbearCrm/FetchXMLSearch/Data/searchform.asmx | - |
yahoo pipes 1.0
| Rule | Path | Comment |
|---|---|---|
| Disallow | /admin/ | - |
| Disallow | /App_Browsers/ | - |
| Disallow | /App_Code/ | - |
| Disallow | /App_Data/ | - |
| Disallow | /App_GlobalResources/ | - |
| Disallow | /bin/ | - |
| Disallow | /Components/ | - |
| Disallow | /Config/ | - |
| Disallow | /contest/ | - |
| Disallow | /controls/ | - |
| Disallow | /DesktopModules/ | - |
| Disallow | /Documentation/ | - |
| Disallow | /HttpModules/ | - |
| Disallow | /images/ | - |
| Disallow | /Install/ | - |
| Disallow | /js/ | - |
| Disallow | /Portals/ | - |
| Disallow | /Providers/ | - |
| Disallow | /Resources/ContentRotator/ | - |
| Disallow | /Resources/ControlPanel/ | - |
| Disallow | /Resources/Dashboard/ | - |
| Disallow | /Resources/FeedBrowser/ | - |
| Disallow | /Resources/OpenForceAd/ | - |
| Disallow | /Resources/Search/ | - |
| Disallow | /Resources/Shared/ | - |
| Disallow | /Resources/SkinWidgets/ | - |
| Disallow | /Resources/TabStrip/ | - |
| Disallow | /Resources/Widgets/ | - |
| Disallow | /Activity-Feed/userId/ | Do not index user profiles |
Comments