legacy.fanbyte.com
robots.txt

Robots Exclusion Standard data for legacy.fanbyte.com

Resource Scan

Scan Details

Site Domain legacy.fanbyte.com
Base Domain fanbyte.com
Scan Status Ok
Last Scan2024-04-25T17:21:48+00:00
Next Scan 2024-05-25T17:21:48+00:00

Last Scan

Scanned2024-04-25T17:21:48+00:00
URL https://legacy.fanbyte.com/robots.txt
Domain IPs 13.33.30.24, 13.33.30.47, 13.33.30.6, 13.33.30.73
Response IP 13.33.30.6
Found Yes
Hash 8bc1c5481a2f56b6df89555f1dfd60c8819434d81ce14e4c76260548418bf8f2
SimHash e14078654911

Groups

*

Rule Path
Disallow /cgi-bin/ratings.pl
Disallow /cgi-bin/prefs-new.pl
Disallow /fsearch.html
Disallow /wiki.html?h=
Disallow /fcluster/gotopost.pl
Disallow /images/

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20