verfwebwinkel.be
robots.txt

Robots Exclusion Standard data for verfwebwinkel.be

Resource Scan

Scan Details

Site Domain verfwebwinkel.be
Base Domain verfwebwinkel.be
Scan Status Ok
Last Scan2025-12-10T19:44:07+00:00
Next Scan 2026-01-09T19:44:07+00:00

Last Scan

Scanned2025-12-10T19:44:07+00:00
URL https://verfwebwinkel.be/robots.txt
Redirect https://www.verfwebwinkel.be/robots.txt
Redirect Domain www.verfwebwinkel.be
Redirect Base verfwebwinkel.be
Domain IPs 104.26.0.107, 104.26.1.107, 172.67.69.251, 2606:4700:20::681a:16b, 2606:4700:20::681a:6b, 2606:4700:20::ac43:45fb
Redirect IPs 104.26.0.107, 104.26.1.107, 172.67.69.251, 2606:4700:20::681a:16b, 2606:4700:20::681a:6b, 2606:4700:20::ac43:45fb
Response IP 104.26.0.107
Found Yes
Hash 8624a36acf7af24ae9bb0a650fef339c746843e22eab5191b74b8bf9dab855c5
SimHash 4909d551c388

Groups

*

Rule Path
Disallow /account/*
Disallow /checkout/*
Disallow /order-confirmation/*
Disallow */api/*
Disallow /*?*
Disallow /*%26*
Disallow /monitoring
Allow *?p=*
Allow *?utm=*
Allow /*?p=*
Allow /single-sitemap?file=*
Allow /*utm_source%3Dtwenga*
Allow /single-sitemap?file=*

adsbot-google
adsbot-google-mobile

Rule Path
Disallow /monitoring?*
Allow *?*

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

blexbot

Rule Path
Disallow /

brightbot 1.0

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

feedbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.verfwebwinkel.be/sitemap.xml