verfwebwinkel.nl
robots.txt

Robots Exclusion Standard data for verfwebwinkel.nl

Resource Scan

Scan Details

Site Domain verfwebwinkel.nl
Base Domain verfwebwinkel.nl
Scan Status Ok
Last Scan2026-04-11T02:49:35+00:00
Next Scan 2026-05-11T02:49:35+00:00

Last Scan

Scanned2026-04-11T02:49:35+00:00
URL https://verfwebwinkel.nl/robots.txt
Redirect https://www.verfwebwinkel.nl/robots.txt
Redirect Domain www.verfwebwinkel.nl
Redirect Base verfwebwinkel.nl
Domain IPs 104.26.4.58, 104.26.5.58, 172.67.68.118, 2606:4700:20::681a:43a, 2606:4700:20::681a:53a, 2606:4700:20::ac43:4476
Redirect IPs 104.26.4.58, 104.26.5.58, 172.67.68.118, 2606:4700:20::681a:43a, 2606:4700:20::681a:53a, 2606:4700:20::ac43:4476
Response IP 104.26.4.58
Found Yes
Hash 05557108af591da04a80fdced417135201c4baedfdad2bb7fd97cf6aa403ac6e
SimHash 6909d151c328

Groups

*

Rule Path
Disallow /account/*
Disallow /checkout/*
Disallow /order-confirmation/*
Disallow */api/*
Disallow /*?*
Disallow /*%26*
Disallow /monitoring
Allow *?p=*
Allow *?utm=*
Allow /*?p=*
Allow /single-sitemap?file=*
Allow /*utm_source%3Dtwenga*
Allow /single-sitemap?file=*

adsbot-google
adsbot-google-mobile

Rule Path
Disallow /monitoring?*
Allow *?*

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

blexbot

Rule Path
Disallow /

brightbot 1.0

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

feedbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.verfwebwinkel.nl/sitemap.xml