bbc.co.uk
robots.txt
Robots Exclusion Standard data for bbc.co.uk
Resource Scan
Scan Details
| Site Domain | bbc.co.uk |
| Base Domain | bbc.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-10-31T17:33:17+00:00 |
| Next Scan | 2024-11-07T17:33:17+00:00 |
Last Scan
| Scanned | 2024-10-31T17:33:17+00:00 |
| URL | https://bbc.co.uk/robots.txt |
| Redirect | https://www.bbc.co.uk/robots.txt |
| Redirect Domain | www.bbc.co.uk |
| Redirect Base | bbc.co.uk |
| Domain IPs | 151.101.0.81, 151.101.128.81, 151.101.192.81, 151.101.64.81, 2a04:4e42:200::81, 2a04:4e42:400::81, 2a04:4e42:600::81, 2a04:4e42::81 |
| Redirect IPs | 151.101.0.81, 151.101.128.81, 151.101.192.81, 151.101.64.81 |
| Response IP | 199.232.44.81 |
| Found | Yes |
| Hash | af01e27def6f8d54cf08a30752044a26a65a8afafea56e97d79b69bdb622ec37 |
| SimHash | 5918e30abeb4 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /bitesize/search$ |
| Disallow | /bitesize/search/ |
| Disallow | /bitesize/search? |
| Disallow | /cbbc/search$ |
| Disallow | /cbbc/search/ |
| Disallow | /cbbc/search? |
| Disallow | /cbeebies/search$ |
| Disallow | /cbeebies/search/ |
| Disallow | /cbeebies/search? |
| Disallow | /chwilio/ |
| Disallow | /chwilio$ |
| Disallow | /chwilio? |
| Disallow | /iplayer/bigscreen/ |
| Disallow | /iplayer/cbbc/episodes/ |
| Disallow | /iplayer/cbbc/search |
| Disallow | /iplayer/cbeebies/episodes/ |
| Disallow | /iplayer/cbeebies/search |
| Disallow | /iplayer/search |
| Disallow | /indepthtoolkit/smallprox$ |
| Disallow | /indepthtoolkit/smallprox/ |
| Disallow | /modules/musicnav/language/ |
| Disallow | /news/0 |
| Disallow | /radio/aod/ |
| Disallow | /radio/aod$ |
| Disallow | /radio/imda |
| Disallow | /radio/player/ |
| Disallow | /radio/player$ |
| Disallow | /search/ |
| Disallow | /search$ |
| Disallow | /search? |
| Disallow | /sport/alpha/ |
| Disallow | /sounds/player/ |
| Disallow | /sounds/player$ |
| Disallow | /ugc$ |
| Disallow | /ugc/ |
| Disallow | /ugcsupport$ |
| Disallow | /ugcsupport/ |
| Disallow | /userinfo/ |
| Disallow | /userinfo |
| Disallow | /food/favourites |
| Disallow | /food/menus/*/shopping-list |
| Disallow | /food/recipes/*/shopping-list |
| Disallow | /food/search*?* |
| Disallow | /sounds/search$ |
| Disallow | /sounds/search/ |
| Disallow | /sounds/search? |
| Disallow | /ws/includes |
| Disallow | /rd/search$ |
| Disallow | /rd/search/ |
| Disallow | /rd/search? |
| Disallow | /things/search$ |
| Disallow | /things/search/ |
| Disallow | /things/search? |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.bbc.co.uk/sitemap.xml |
| sitemap | https://www.bbc.co.uk/sitemaps/https-index-uk-archive.xml |
| sitemap | https://www.bbc.co.uk/sitemaps/https-index-uk-news.xml |
| sitemap | https://www.bbc.co.uk/food/sitemap.xml |
| sitemap | https://www.bbc.co.uk/bitesize/sitemap/sitemapindex.xml |
| sitemap | https://www.bbc.co.uk/teach/sitemap/sitemapindex.xml |
| sitemap | https://www.bbc.co.uk/sitemaps/https-index-uk-archive_video.xml |
| sitemap | https://www.bbc.co.uk/sitemaps/https-index-uk-video.xml |
| sitemap | https://www.bbc.co.uk/sitemaps/sitemap-uk-ws-topics.xml |
| sitemap | https://www.bbc.co.uk/sport/sitemap.xml |
| sitemap | https://www.bbc.co.uk/sitemaps/sitemap-uk-topics.xml |
| sitemap | https://www.bbc.co.uk/ideas/sitemap.xml |
| sitemap | https://www.bbc.co.uk/tiny-happy-people/sitemap/sitemapindex.xml |
Comments