baars-bloemhoff.nl
robots.txt

Robots Exclusion Standard data for baars-bloemhoff.nl

Resource Scan

Scan Details

Site Domain baars-bloemhoff.nl
Base Domain baars-bloemhoff.nl
Scan Status Ok
Last Scan2025-10-24T05:24:25+00:00
Next Scan 2025-11-07T05:24:25+00:00

Last Scan

Scanned2025-10-24T05:24:25+00:00
URL https://www.baars-bloemhoff.nl/robots.txt
Domain IPs 13.69.68.11
Response IP 13.69.68.11
Found Yes
Hash 5d031792498bc25af4d298b2a430375524ceaa8ba1886b97408c5cb7e4f5c691
SimHash 524cd842dcbb

Groups

*

Rule Path
Disallow /INTERSHOP/web/WFS/SLDSystem/
Disallow /INTERSHOP/web/BOS/SMC/
Disallow /monsters
Disallow /inloggen
Disallow /uitloggen
Disallow /winkelmand
Disallow /INTERSHOP/web/WFS/stihoGroep-BaarsEnBloemhoff-Site/nl_NL/-/EUR/ViewCart
Disallow /viewdata
Disallow /INTERSHOP/web/WFS/stihoGroep-BaarsEnBloemhoff-Site/nl_NL/-/EUR/ViewCheckout
Disallow /account
Disallow /INTERSHOP/web/WFS/stihoGroep-BaarsEnBloemhoff-Site/nl_NL/-/EUR/DSG_AddressesM3
Disallow /INTERSHOP/web/WFS/stihoGroep-BaarsEnBloemhoff-Site/nl_NL/-/EUR/ViewAccount
Disallow /INTERSHOP/web/WFS/stihoGroep-BaarsEnBloemhoff-Site/nl_NL/-/EUR/ViewOrder
Disallow /INTERSHOP/web/WFS/stihoGroep-BaarsEnBloemhoff-Site/nl_NL/-/EUR/ViewMostPurchasedItems
Disallow /INTERSHOP/web/WFS/stihoGroep-BaarsEnBloemhoff-Site/nl_NL/-/EUR/ViewParametricSearch-SimpleOfferSearch
Disallow /INTERSHOP/web/WFS/stihoGroep-BaarsEnBloemhoff-Site/nl_NL/-/EUR/ViewSuggestSearch-Suggest
Disallow /INTERSHOP/web/WFS/stihoGroep-BaarsEnBloemhoff-Site/nl_NL/-/EUR/GetProductData
Disallow /*SearchParameter%3D
Disallow /*PageableID%3D
Disallow /filter
Disallow /zoeken

seokicks-robot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot/7~bl

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

kingkevinbot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

cazoodlebot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.baars-bloemhoff.nl/sitemap.xml

Comments

  • will be used by all Yandex robots
  • contentking
  • new agent exclusions 07-11-2022
  • exclude AI bots