voervergelijk.nl
robots.txt

Robots Exclusion Standard data for voervergelijk.nl

Resource Scan

Scan Details

Site Domain voervergelijk.nl
Base Domain voervergelijk.nl
Scan Status Ok
Last Scan6/6/2025, 1:15:03 AM
Next Scan 6/13/2025, 1:15:03 AM

Last Scan

Scanned6/6/2025, 1:15:03 AM
URL https://voervergelijk.nl/robots.txt
Redirect https://www.voervergelijk.nl/robots.txt
Redirect Domain www.voervergelijk.nl
Redirect Base voervergelijk.nl
Domain IPs 104.21.50.200, 172.67.166.143, 2606:4700:3031::ac43:a68f, 2606:4700:3034::6815:32c8
Redirect IPs 104.21.50.200, 172.67.166.143, 2606:4700:3031::ac43:a68f, 2606:4700:3034::6815:32c8
Response IP 104.21.50.200
Found Yes
Hash e64c5bd07700b270bd5f55ec051632b89e035f70c2f6257146aecb1929160e56
SimHash 54041bd3e7b5

Groups

*

Rule Path
Disallow /captcha/
Disallow /ajax/
Disallow /classes/
Disallow /cms/
Disallow /js/
Disallow /module/
Disallow /reactie/
Disallow /suggestie/

adsbot-google
amazonbot
anthropic-ai
applebot-extended
bytespider
ccbot
chatgpt-user
claudebot
claude-web
cohere-ai
diffbot
facebookbot
friendlycrawler
google-extended
googleother
gptbot
img2dataset
omgili
omgilibot
peer39_crawler
peer39_crawler/1.0
perplexitybot
youbot

Rule Path
Disallow /