engrxiv.org
robots.txt

Robots Exclusion Standard data for engrxiv.org

Resource Scan

Scan Details

Site Domain engrxiv.org
Base Domain engrxiv.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-05T21:44:57+00:00
Next Scan 2025-12-04T21:44:57+00:00

Last Successful Scan

Scanned2025-04-16T19:25:27+00:00
URL https://engrxiv.org/robots.txt
Domain IPs 204.187.13.48
Response IP 204.187.13.48
Found Yes
Hash 7c72f777548652ec4207c2f5f742bf30ff90e24e45eb5ff6f03cb22c920b7b61
SimHash 2138dcc0c1b6

Groups

*

Rule Path
Disallow /cache/

Other Records

Field Value
crawl-delay 60