structureddata.org
robots.txt

Robots Exclusion Standard data for structureddata.org

Resource Scan

Scan Details

Site Domain structureddata.org
Base Domain structureddata.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-02-07T14:44:23+00:00
Next Scan 2026-05-08T14:44:23+00:00

Last Successful Scan

Scanned2024-06-24T22:51:35+00:00
URL https://structureddata.org/robots.txt
Domain IPs 104.21.20.159, 172.67.193.47, 2606:4700:3033::6815:149f, 2606:4700:3034::ac43:c12f
Response IP 172.67.193.47
Found Yes
Hash 0ad677ac171cd97bc25f84d1ba6f5658f2472ba5afc395d7ca4d450c4b766492
SimHash 6a008a4347b2

Groups

*

Rule Path
Allow /*
Disallow /admin/*
Disallow /api/*

Other Records

Field Value
sitemap https://structureddata.org/sitemap.xml

Warnings

  • `host` is not a known field.