laptophub.uk
robots.txt

Robots Exclusion Standard data for laptophub.uk

Resource Scan

Scan Details

Site Domain laptophub.uk
Base Domain laptophub.uk
Scan Status Ok
Last Scan2026-02-04T03:01:24+00:00
Next Scan 2026-03-06T03:01:24+00:00

Last Scan

Scanned2026-02-04T03:01:24+00:00
URL https://laptophub.uk/robots.txt
Domain IPs 104.26.0.57, 104.26.1.57, 172.67.68.45, 2606:4700:20::681a:139, 2606:4700:20::681a:39, 2606:4700:20::ac43:442d
Response IP 104.26.1.57
Found Yes
Hash eb0f13f5db863ef113d33b86f288b0929bd09b049ee609dc5199b553e2b39abb
SimHash d23cf807e70b

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

*

Rule Path
Disallow /app/
Disallow /cgi-bin/
Disallow /downloader/
Disallow /errors/
Disallow /includes/
Disallow /lib/
Disallow /magento/
Disallow /pkginfo/
Disallow /report/
Disallow /scripts/
Disallow /shell/
Disallow /stats/
Disallow /var/
Disallow /index.php/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalog/product/gallery/
Disallow /catalogsearch/
Disallow /checkout/
Disallow /control/
Disallow /contacts/
Disallow /customer/
Disallow /customize/
Disallow /newsletter/
Disallow /poll/
Disallow /review/
Disallow /sendfriend/
Disallow /tag/
Disallow /wishlist/
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /STATUS.txt
Disallow /*?SID=
Disallow /*?manufacturer=
Disallow /*%26sinch_attr_

screaming frog seo spider

Rule Path
Disallow /*.gif$
Disallow /*.jpg$
Disallow /*.png$
Disallow /*.bmp$
Disallow /*.xml$
Disallow /*.js$
Disallow /*.css$
Disallow /*.php$
Disallow /*?p=*
Disallow /*?limit=all
Disallow /*?PHPSESSID=
Disallow /*?price=
Disallow /*%26price%3D
Disallow /*?color=
Disallow /*%26color%3D
Disallow /*?material=
Disallow /*%26material%3D
Disallow /*?size=
Disallow /*%26size%3D

semrushbot
semrushbot-sa
semrushbot-ba
ahrefsbot
blexbot
baiduspider
baiduspider-image
baiduspider-video
mj12bot
dotbot
ezooms
ia_archiver
rogerbot
moget
ichiro
sogou spider
youdaobot
naverbot
yeti
nutch
pricebot
pinterestbot
claudebot
amazonbot
gptbot
petalbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /search/
Disallow /catalogsearch/result/?

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.laptophub.uk/sitemap/sitemap.xml

Comments

  • Screaming Frog
  • Aggressive, spammy, or irrelevant bots
  • Moderate Bingbot
  • Optional: Allow only Idealo if desired
  • User-agent: idealo-bot
  • Disallow:
  • Optional: Uncomment if Yandex needs blocking
  • User-agent: Yandex
  • Disallow: /