multipharma.be
robots.txt

Robots Exclusion Standard data for multipharma.be

Resource Scan

Scan Details

Site Domain multipharma.be
Base Domain multipharma.be
Scan Status Ok
Last Scan2026-02-17T10:04:03+00:00
Next Scan 2026-03-03T10:04:03+00:00

Last Scan

Scanned2026-02-17T10:04:03+00:00
URL https://www.multipharma.be/robots.txt
Domain IPs 104.16.212.59, 190.93.246.34
Response IP 104.16.212.59
Found Yes
Hash 53f06affbc01d1527f2c8dd22b0535f6f9452984c09e8fe92f9f01720fc5ddc8
SimHash c17bc14463fa

Groups

*

Rule Path
Disallow /on/demandware.store/*
Disallow *Login-*
Disallow *Register-*
Disallow *Wishlist-*
Disallow *Cart-*
Disallow *Checkout-*
Disallow *Account-*
Disallow *Order-*
Disallow /*null*
Allow /*?dwvar
Allow /*utm_medium%3Dmerchant
Disallow /*pmin*
Disallow /*pmax*
Disallow /*srule%3D*
Disallow /*?q=*
Disallow /*prefv*%3D*
Disallow /*prefn*%3D*

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

mj12bot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.multipharma.be/sitemap_index.xml

Comments

  • Transactional and Content
  • Search and Query Params
  • Slow crawling rate for some agents
  • Disallow aggressive agents
  • Sitemaps

Warnings

  • 2 invalid lines.