Log In Sign Up

spiderdao.io
robots.txt

Robots Exclusion Standard data for spiderdao.io

Resource Scan

Scan Details

Site Domain spiderdao.io
Base Domain spiderdao.io
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-22T06:21:54+00:00
Next Scan 2025-11-05T06:21:54+00:00

Last Successful Scan

Scanned2025-09-14T05:12:41+00:00
URL https://spiderdao.io/robots.txt
Domain IPs 104.21.25.18, 172.67.222.3, 2606:4700:3032::6815:1912, 2606:4700:3034::ac43:de03
Response IP 172.67.222.3
Found Yes
Hash 6b69b34929ecefe44acf1f95d91e080a6f573c9c1d68f5433dcdc1b196ada9b8
SimHash 60204a608b33

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Back to top

Warnings

  • 1 invalid line.
  • `<b>notice</b>` is not a known field.
Back to top