afdi.us
robots.txt

Robots Exclusion Standard data for afdi.us

Resource Scan

Scan Details

Site Domain afdi.us
Base Domain afdi.us
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-17T11:57:42+00:00
Next Scan 2026-02-15T11:57:42+00:00

Last Successful Scan

Scanned2025-10-18T07:58:32+00:00
URL https://afdi.us/robots.txt
Domain IPs 185.151.30.213
Response IP 185.151.30.213
Found Yes
Hash 2ee1e5766ddc7fd5f6c6cda16f2923d96068f123ba334fe8e65d252199fc0982
SimHash 21a88ec3ffb1

Groups

*

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

Comments

  • Default robots.txt file (which asks all bots to crawl slowly but still index everything).