bgstechnic.com
robots.txt

Robots Exclusion Standard data for bgstechnic.com

Resource Scan

Scan Details

Site Domain bgstechnic.com
Base Domain bgstechnic.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-06T04:39:57+00:00
Next Scan 2026-03-06T04:39:57+00:00

Last Successful Scan

Scanned2025-07-16T12:58:33+00:00
URL https://bgstechnic.com/robots.txt
Redirect https://www.bgstechnic.com/robots.txt
Redirect Domain www.bgstechnic.com
Redirect Base bgstechnic.com
Domain IPs 104.26.10.74, 104.26.11.74, 172.67.74.158, 2606:4700:20::681a:a4a, 2606:4700:20::681a:b4a, 2606:4700:20::ac43:4a9e
Redirect IPs 104.26.10.74, 104.26.11.74, 172.67.74.158, 2606:4700:20::681a:a4a, 2606:4700:20::681a:b4a, 2606:4700:20::ac43:4a9e
Response IP 104.26.10.74
Found Yes
Hash 19fae3c589c60bdf074151974398b3cd04fbcc9f343da0467d107d373fe70cd1
SimHash a61ccdd27abd

Groups

*

Rule Path
Disallow /admin/
Disallow /Core/
Disallow /tmp/
Disallow /cache/
Disallow /views/
Disallow /Setup/
Disallow /log/
Disallow /newsletter/
Disallow /en/newsletter/
Disallow /index.php?cl=newsletter
Disallow /AGB/
Disallow /en/Terms-and-Conditions/
Disallow /warenkorb/
Disallow /en/cart/
Disallow /index.php?cl=basket
Disallow /mein-konto/
Disallow /en/my-account/
Disallow /index.php?cl=account
Disallow /mein-merkzettel/
Disallow /en/my-wishlist/
Disallow /index.php?cl=account_noticelist
Disallow /mein-wunschzettel/
Disallow /en/my-gift-registry/
Disallow /index.php?cl=account_wishlist
Disallow /konto-eroeffnen/
Disallow /en/open-account/
Disallow /index.php?cl=register
Disallow /passwort-vergessen/
Disallow /en/forgot-password/
Disallow /index.php?cl=forgotpwd
Disallow /index.php?cl=moredetails
Disallow /index.php?cl=review
Disallow /index.php?cl=search
Disallow /EXCEPTION_LOG.txt
Disallow /*?cl=newsletter
Disallow /*%26cl%3Dnewsletter
Disallow /*?cl=basket
Disallow /*%26cl%3Dbasket
Disallow /*?cl=account
Disallow /*%26cl%3Daccount
Disallow /*?cl=account_noticelist
Disallow /*%26cl%3Daccount_noticelist
Disallow /*?cl=account_wishlist
Disallow /*%26cl%3Daccount_wishlist
Disallow /*?cl=register
Disallow /*%26cl%3Dregister
Disallow /*?cl=forgotpwd
Disallow /*%26cl%3Dforgotpwd
Disallow /*?cl=moredetails
Disallow /*%26cl%3Dmoredetails
Disallow /*?cl=review
Disallow /*%26cl%3Dreview
Disallow /*?cl=search
Disallow /*%26cl%3Dsearch
Disallow /*%26fnc%3Dtobasket
Disallow /*%26fnc%3Dtocomparelist
Disallow /*%26addcompare%3D
Disallow /*/sid/
Disallow /*?sid=
Disallow /*%26sid%3D
Disallow /*?cur=
Disallow /*%26cur
Disallow /*?renderPdf=1
Disallow /*renderPdf%3D1
Disallow /*?renderPdf=1&aid=*
Disallow /*renderPdf%3D1%26aid%3D*
Disallow /*renderPdf%3D1*
Disallow /*?renderpdf=1
Disallow /*renderpdf%3D1
Disallow /*?renderpdf=1&aid=*
Disallow /*renderpdf%3D1%26aid%3D*
Disallow /*renderpdf%3D1*

megaindex.ru

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

webwiki

Rule Path
Disallow /

kaloogabot

Rule Path
Disallow /

webinatorbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

npbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ia_archiver/1.6

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

grub

Rule Path
Disallow /

copyrightcheck

Rule Path
Disallow /

teleport*

Rule Path
Disallow /

webwhacker*

Rule Path
Disallow /

webzip*

Rule Path
Disallow /

net attache*

Rule Path
Disallow /

sitesnagger*

Rule Path
Disallow /

httrack*

Rule Path
Disallow /

webcapture*

Rule Path
Disallow /

websauger*

Rule Path
Disallow /

webreaper*

Rule Path
Disallow /

roverbot*

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

komodiabot

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

domaincrawler*

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

Comments

  • wildcards at the end, because of some crawlers see it as errors