sailwx.info
robots.txt

Robots Exclusion Standard data for sailwx.info

Resource Scan

Scan Details

Site Domain sailwx.info
Base Domain sailwx.info
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-04-08T17:58:40+00:00
Next Scan 2024-07-07T17:58:40+00:00

Last Successful Scan

Scanned2022-05-14T15:08:09+00:00
URL https://sailwx.info/robots.txt
Response IP 23.253.237.94
Found Yes
Hash 68bf3605b4df3cb2eefa66975b4a3e56b1d7d949bc4221cd6778f3ce56a607bb
SimHash 221dc4514331

Groups

*

Rule Path
Disallow /tidesBeta
Disallow /test
Disallow /shiptrack/shipdump.phtml
Disallow /shiptrack/shipdump.phtml*
Disallow /shiptrack/shipdump.phtml?call*
Disallow /shiptrack/shiplocations.phtml*

yandex

Rule Path
Disallow /shiptrack/shiplocations.phtml

Other Records

Field Value
crawl-delay 5

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

e-societyrobot

Rule Path
Disallow /

slurp

Rule Path
Disallow /wxobs
Disallow shiptrack/shiplocations.phtml

Other Records

Field Value
crawl-delay 10

twiceler

Rule Path
Disallow /wxobs

exabot

Rule Path
Disallow /

expo9

Rule Path
Disallow /

jobdiggerspider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

openwebspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /