usfl.com
robots.txt

Robots Exclusion Standard data for usfl.com

Resource Scan

Scan Details

Site Domain usfl.com
Base Domain usfl.com
Scan Status Ok
Last Scan2024-09-23T22:27:57+00:00
Next Scan 2024-09-30T22:27:57+00:00

Last Scan

Scanned2024-09-23T22:27:57+00:00
URL https://usfl.com/robots.txt
Domain IPs 157.7.44.217
Response IP 157.7.44.217
Found Yes
Hash 29da4ea3d671dae5da9dd5a4726c5cf4b4010dab04657d1ab23f98134596656e
SimHash c43cdc7022cb

Groups

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5