leiden.nl
robots.txt

Robots Exclusion Standard data for leiden.nl

Resource Scan

Scan Details

Site Domain leiden.nl
Base Domain leiden.nl
Scan Status Ok
Last Scan2024-10-02T07:17:02+00:00
Next Scan 2024-10-16T07:17:02+00:00

Last Scan

Scanned2024-10-02T07:17:02+00:00
URL https://leiden.nl/robots.txt
Redirect https://gemeente.leiden.nl/robots.txt
Redirect Domain gemeente.leiden.nl
Redirect Base leiden.nl
Domain IPs 2001:9a8:a6:0:87:233:198:3, 87.233.198.3
Redirect IPs 2001:9a8:a6:0:87:233:198:3, 87.233.198.3
Response IP 87.233.198.3
Found Yes
Hash 29692dbfc3c3a34a920fe9fc4138cf0e7c2eae8588cc214240c8391cc6f16cd5
SimHash d00443024a33

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

ahrefsbot
baiduspider
baiduspider-video
baiduspider-image
dotbot
ichiro
mauibot
mj12bot
moget
naverbot
petalbot
semrushbot
seznambot
sogou spider
yandex
yeti
youdaobot

Rule Path
Disallow /