sawiday.be
robots.txt

Robots Exclusion Standard data for sawiday.be

Resource Scan

Scan Details

Site Domain sawiday.be
Base Domain sawiday.be
Scan Status Ok
Last Scan2024-05-23T22:57:01+00:00
Next Scan 2024-06-22T22:57:01+00:00

Last Scan

Scanned2024-05-23T22:57:01+00:00
URL https://sawiday.be/robots.txt
Redirect https://www.sawiday.be/robots.txt
Redirect Domain www.sawiday.be
Redirect Base sawiday.be
Domain IPs 104.26.12.160, 104.26.13.160, 172.67.73.92, 2606:4700:20::681a:ca0, 2606:4700:20::681a:da0, 2606:4700:20::ac43:495c
Redirect IPs 104.26.12.160, 104.26.13.160, 172.67.73.92, 2606:4700:20::681a:ca0, 2606:4700:20::681a:da0, 2606:4700:20::ac43:495c
Response IP 172.67.73.92
Found Yes
Hash b6de9b0eade0d47a66f236095e574ed6253ff288bf1bc115784f52e01cda5cc1
SimHash 502fc460e1bf

Groups

*
*

Rule Path
Disallow /nl-be/zoeken/*
Disallow /nl-be/vergelijken/*
Disallow /nl-be/winkelwagen/*
Disallow /nl-be/img.php?
Disallow /nl-be/merkproduct.php?
Disallow /nl-be/*addproductreview.php?
Disallow /nl-be/locationpopup/render-popup

*

Rule Path
Disallow /fr-be/chercher/*
Disallow /fr-be/comparer/*
Disallow /fr-be/panier/*

*

Rule Path
Disallow /*?minprice=
Disallow /*?maxprice=
Disallow /*?breedte=
Disallow /*?tt=
Disallow /*?search=
Disallow /*?subject%2Bprefill=
Disallow /nl-be/verlanglijst/
Disallow /fr-be/liste-denvies/
Disallow /*rolcontainers.
Disallow /*ik-wil-meer-weten.php?
Disallow *?swbe-popup=1

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

feedbot

Rule Path
Disallow /

brightbot 1.0

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.sawiday.be/nl-be/sitemap/
sitemap https://www.sawiday.be/fr-be/sitemap/

Comments

  • site nl-be
  • site fr-be
  • site generic
  • specific bots