Log In Sign Up

wur.nl
robots.txt

Robots Exclusion Standard data for wur.nl

Resource Scan

Scan Details

Site Domain wur.nl
Base Domain wur.nl
Scan Status Ok
Last Scan2024-06-07T01:29:48+00:00
Next Scan 2024-06-21T01:29:48+00:00

Last Scan

Scanned2024-06-07T01:29:48+00:00
URL https://wur.nl/robots.txt
Redirect https://www.wur.nl/robots.txt
Redirect Domain www.wur.nl
Redirect Base wur.nl
Domain IPs 137.224.9.30, 2001:610:a38:4200::1000
Redirect IPs 2600:1413:b000:1a::17d7:74a, 2600:1413:b000:1a::17d7:758, 96.17.96.17, 96.17.96.8
Response IP 23.59.168.130
Found Yes
Hash 404c91a907f22458bb22f722f51f787a3057766c5dde9427772c5c8491f8c7bf
SimHash eb089a328bb3

Groups

wbsearchbot

Product Comment
wbsearchbot zie http://www.warebay.com/bot.html en SR112667
Rule Path
Disallow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top

Other Records

Field Value
sitemap https://www.wur.nl/sitemap.xml.gz
Back to top