securea.mlb.com
robots.txt

Robots Exclusion Standard data for securea.mlb.com

Resource Scan

Scan Details

Site Domain securea.mlb.com
Base Domain mlb.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-03-28T20:27:25+00:00
Next Scan 2024-06-26T20:27:25+00:00

Last Successful Scan

Scanned2023-08-24T14:34:36+00:00
URL https://securea.mlb.com/robots.txt
Domain IPs 23.209.46.81, 23.209.46.90
Response IP 104.91.68.51
Found Yes
Hash 9a8eeeaff47bbb7aa3ba6dc6757f79876b25ef272c77d9b807e76e53967a2eb1
SimHash ed4e45014691

Groups

truveo

Rule Path
Disallow /

msnbot

Rule Path
Disallow /video/index.jsp*

Other Records

Field Value
crawl-delay 6

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 6

yahooseeker

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

daumoa

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 12

voilabot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

friendfeedbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

unwindfetchor

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

*

Rule Path
Disallow /mlb/gameday/
Disallow /team/player_career.jsp
Disallow /team/player_news.jsp
Disallow /lookup/
Disallow /hou/fan_forum/mascot_appearance_form.jsp
Disallow /ws/search/
Disallow /ana/
Disallow /ari/
Disallow /atl/
Disallow /bal/
Disallow /bos/
Disallow /chc/
Disallow /cin/
Disallow /cle/
Disallow /col/
Disallow /cws/
Disallow /det/
Disallow /hou/
Disallow /kc/
Disallow /la/
Disallow /mia/
Disallow /mil/
Disallow /min/
Disallow /nym/
Disallow /nyy/
Disallow /oak/
Disallow /phi/
Disallow /pit/
Disallow /sd/
Disallow /sea/
Disallow /sf/
Disallow /stl/
Disallow /tb/
Disallow /tex/
Disallow /tor/
Disallow /was/
Disallow /es/ana/
Disallow /es/ari/
Disallow /es/atl/
Disallow /es/bal/
Disallow /es/bos/
Disallow /es/chc/
Disallow /es/cin/
Disallow /es/cle/
Disallow /es/col/
Disallow /es/cws/
Disallow /es/det/
Disallow /es/hou/
Disallow /es/kc/
Disallow /es/la/
Disallow /es/mia/
Disallow /es/mil/
Disallow /es/min/
Disallow /es/nym/
Disallow /es/nyy/
Disallow /es/oak/
Disallow /es/phi/
Disallow /es/pit/
Disallow /es/sd/
Disallow /es/sea/
Disallow /es/sf/
Disallow /es/stl/
Disallow /es/tb/
Disallow /es/tex/
Disallow /es/tor/
Disallow /es/was/
Disallow /scripts/
Disallow /shared/scripts/