kasparovru.com
robots.txt

Robots Exclusion Standard data for kasparovru.com

Resource Scan

Scan Details

Site Domain kasparovru.com
Base Domain kasparovru.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-21T13:43:38+00:00
Next Scan 2024-06-20T13:43:38+00:00

Last Successful Scan

Scanned2024-04-22T13:42:10+00:00
URL https://kasparovru.com/robots.txt
Domain IPs 204.246.191.125, 204.246.191.24, 204.246.191.7, 204.246.191.85
Response IP 18.165.171.117
Found Yes
Hash 30cef91a8f909a97b12e6cd77eadcf6e7d1c39e2ed2663d0ce7516a1e06aa8be
SimHash 6550dc6243b0

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

Warnings

  • 1 invalid line.
  • `host` is not a known field.