julianatoren.nl
robots.txt

Robots Exclusion Standard data for julianatoren.nl

Resource Scan

Scan Details

Site Domain julianatoren.nl
Base Domain julianatoren.nl
Scan Status Ok
Last Scan2024-10-09T09:19:04+00:00
Next Scan 2024-11-08T09:19:04+00:00

Last Scan

Scanned2024-10-09T09:19:04+00:00
URL https://julianatoren.nl/robots.txt
Domain IPs 2001:4038:0:17::23, 83.137.17.23
Response IP 83.137.17.23
Found Yes
Hash 86c19c1846ba7ff622a402afb2495a8fc3a1d3b753ae615abc78796f5755805a
SimHash 1a70d8802931

Groups

*

Rule Path
Disallow /author/
Disallow /wp-admin/
Disallow /wp-includes/

mj12bot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

bspider

Rule Path
Disallow /

Comments

  • Crawlers