wirly.be
robots.txt

Robots Exclusion Standard data for wirly.be

Resource Scan

Scan Details

Site Domain wirly.be
Base Domain wirly.be
Scan Status Ok
Last Scan2025-09-10T21:34:01+00:00
Next Scan 2025-09-17T21:34:01+00:00

Last Scan

Scanned2025-09-10T21:34:01+00:00
URL https://wirly.be/robots.txt
Domain IPs 185.220.172.5, 2a06:2ec0:1:e::147
Response IP 185.220.172.5
Found Yes
Hash 9832d97334ebfb70a71bf841424a2409aa9da50996acdfa7f62a02783a389bd6
SimHash 21748c755691

Groups

*

Rule Path
Disallow /oc-content/downloads/
Disallow /oc-content/languages/
Disallow /oc-content/uploads/temp/
Disallow /oc-includes/vendor/

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /oc-content/downloads/
Disallow /oc-content/languages/
Disallow /oc-content/uploads/HTML/
Disallow /oc-content/uploads/temp/
Disallow /oc-content/plugins/
Disallow /oc-includes/vendor/

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-video

Rule Path
Disallow

googlebot-news

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

baiduspider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.wirly.be/sitemap.xml
sitemap https://www.wirly.be/sitemap_blog.xml
sitemap https://wirly.be/sitemap.xml
sitemap https://www.wirly.be/sitemap.xml
sitemap https://www.wirly.be/sitemap_blog.xml

Comments

  • Algemene regels voor alle bots behalve Google
  • Specifieke bots die zeker de crawl-delay moeten volgen
  • Volledige toegang voor Google bots
  • Volledige blokkade van bepaalde bots