Log In Sign Up

broadband.co.uk
robots.txt

Robots Exclusion Standard data for broadband.co.uk

Resource Scan

Scan Details

Site Domain broadband.co.uk
Base Domain broadband.co.uk
Scan Status Ok
Last Scan2025-04-17T07:51:14+00:00
Next Scan 2025-05-17T07:51:14+00:00

Last Scan

Scanned2025-04-17T07:51:14+00:00
URL https://broadband.co.uk/robots.txt
Redirect https://www.broadband.co.uk/robots.txt
Redirect Domain www.broadband.co.uk
Redirect Base broadband.co.uk
Domain IPs 104.26.6.48, 104.26.7.48, 172.67.69.248, 2606:4700:20::681a:630, 2606:4700:20::681a:730, 2606:4700:20::ac43:45f8
Redirect IPs 104.26.6.48, 104.26.7.48, 172.67.69.248, 2606:4700:20::681a:630, 2606:4700:20::681a:730, 2606:4700:20::ac43:45f8
Response IP 104.26.7.48
Found Yes
Hash 7f84ded0707260cc480fa827da059286965eb5fcc21ff06ad69d9312155881fa
SimHash a362ae7f965a

Groups

trovitbot

Rule Path
Disallow /

magpie-crawler/1.1

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

*

Rule Path
Disallow /broadband/out/
Disallow /broadband/out
Disallow /mobilebroadband/out/
Disallow /mobilebroadband/out
Disallow /smartphones/out/
Disallow /smartphones/out
Disallow /accessories/out/
Disallow /accessories/out
Disallow /broadband/partner/sitewise/out/
Disallow /mobilebroadband/partner/sitewise/out/
Disallow /broadband/partner/pcadvisor/out/
Disallow /mobilebroadband/partner/pcadvisor/out/
Disallow /broadband/partner/whatmobile/out/
Disallow /mobilebroadband/partner/whatmobile/out/
Disallow /broadband/partner/moneymagpie/out/
Disallow /mobilebroadband/partner/moneymagpie/out/
Disallow /broadband/partner/mbblog/out/
Disallow /mobilebroadband/partner/mbblog/out/
Disallow /packed/
Disallow /packed
Disallow /strawpoll/
Disallow /strawpoll
Disallow /moneysaver.xml
Disallow /gt
Disallow /pcadvisor
Disallow /pcadvisor/
Disallow /mobilebroadband/partner
Disallow /broadband/partner
Disallow /mobilebroadband/partner/
Disallow /broadband/partner/
Disallow /tracking*
Disallow /upload
Disallow /search
Disallow /backgrounds
Disallow /buttons
Disallow /broadband/personalise
Disallow *?*showAll*
Disallow *?*network*
Disallow *?*order*
Disallow *?*model*
Disallow *?*id*
Disallow /preferences
Disallow /O4CBgFQx
Disallow /thankyoucomp
Disallow /thanks
Disallow /media-assets*
Disallow /ajax/
Disallow /popups/
Disallow /report-abuse/
Disallow /bounty
Disallow /atg
Disallow /thane
Disallow /vectis
Disallow /offers
Disallow /offers/*
Disallow /cashback
Disallow /cashback/*
Disallow *tel%3A*
Disallow /ajax/*
Disallow /broadband/tools/speedtest?*
Disallow /laptops
Disallow /laptops/*
Disallow /best
Disallow /best/*
Disallow /local/
Disallow /forums/archive/
Disallow /forums/sitemap/
Disallow /visit/
Disallow /move/
Disallow /widget/
Disallow /widgets/
Disallow /postcode_widget
Disallow /check/postcode/
Disallow /speedtest-poll-results
Disallow /embedded
Disallow /smartphones*
Allow /smartphones/help/the-application-guide-smartphone-apps
Allow /smartphones/help/smartphone-dictionary-glossary
Allow /smartphones/help/guide-to-smartphone-connectivity-broadband-wifi
Allow /mobilebroadband/pay-as-you-go?showAll=1
Allow /mobilebroadband/3g-mobile-hotspot?showAll=1
Allow /mobilebroadband/dongle?module=mobilebroadband&order=downloadLimit
Allow /mobilebroadband?order=downloadLimit
Allow /broadband/home?network=&order=
Allow /broadband/deals?network=32&order=
Allow /broadband/fibre-optic?network=33&order=
Allow /deals?order=monthlyCost
Allow /mobilebroadband/pay-as-you-go?network=0&order=setupCost
Allow /broadband/free-broadband?order=&filters%5Bspeed%5D=20000
Allow /mobilebroadband/dongle?module=mobilebroadband&order=downloadLimit
Allow /move/?deal_url=https%3A%2F%2Fwww.broadbandgenie.co.uk%2Ftracker%2FBTL%2Fbroadband%2Fproduct%2Flead%2F970%3Fpos%3D19%26source%3Dpopular_deals%26partner%3Dpb3-cb
Back to top

Other Records

Field Value
sitemap https://www.broadband.co.uk/sitemap.xml
Back to top