dol-in.org
robots.txt

Robots Exclusion Standard data for dol-in.org

Resource Scan

Scan Details

Site Domain dol-in.org
Base Domain dol-in.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-02-12T16:19:58+00:00
Next Scan 2026-04-13T16:19:58+00:00

Last Successful Scan

Scanned2023-06-02T02:47:46+00:00
URL https://dol-in.org/robots.txt
Domain IPs 35.171.57.87
Response IP 35.171.57.87
Found Yes
Hash f503c734963e9a9f2a76e5700caabf45ff6af263fedb5d60c0c9f26f35cec560
SimHash 6354dc134311

Groups

mj12bot

Rule Path
Disallow /

*

Rule Path
Disallow
Allow /

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://dol-in.org/sitemap2446.xml