gfherald.com
robots.txt

Robots Exclusion Standard data for gfherald.com

Resource Scan

Scan Details

Site Domain gfherald.com
Base Domain gfherald.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-11-07T16:58:48+00:00
Next Scan 2026-02-05T16:58:48+00:00

Last Successful Scan

Scanned2022-02-02T01:35:56+00:00
URL http://gfherald.com/robots.txt
Redirect https://www.grandforksherald.com/robots.txt
Redirect Domain www.grandforksherald.com
Redirect Base grandforksherald.com
Response IP 52.84.251.110
Found Yes
Hash 469faaa716536fe47459db1abba0fb8b8924660ab87c6c012cf0f88bd4f7a161
SimHash 6c30dc606f92

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Warnings

  • 2 invalid lines.