bluejays.mlb.com
robots.txt
Robots Exclusion Standard data for bluejays.mlb.com
Resource Scan
Scan Details
Site Domain | bluejays.mlb.com |
Base Domain | mlb.com |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Server returned a client error. |
Last Scan | 2024-08-29T08:41:54+00:00 |
Next Scan | 2024-11-27T08:41:54+00:00 |
Last Successful Scan
Scanned | 2024-02-02T07:35:11+00:00 |
URL | http://bluejays.mlb.com/robots.txt |
Redirect | https://www.mlb.com/robots.txt |
Redirect Domain | www.mlb.com |
Redirect Base | mlb.com |
Domain IPs | 34.102.169.19 |
Redirect IPs | 104.18.38.42, 172.64.149.214, 2606:4700:4400::6812:262a, 2606:4700:4400::ac40:95d6 |
Response IP | 172.64.149.214 |
Found | Yes |
Hash | 52640b6e0794688e1d7e0ddb9e1369c8d7695ba11d6bf578aee05df172e67d36 |
SimHash | 91959b2f843b |
Groups
*
Rule | Path |
---|---|
Disallow | /api/ |
Disallow | /app/ |
Disallow | /embed/ |
Disallow | /en/ |
Disallow | /mlb/ |
Disallow | /share/ |
Disallow | /tokens/ |
Disallow | /tv/ |
Disallow | /web/ |
Disallow | /legacy |
Disallow | /beta |
Disallow | /es/legacy |
Disallow | /es/beta |
Disallow | /search |
Disallow | /angels/search |
Disallow | /astros/search |
Disallow | /athletics/search |
Disallow | /bluejays/search |
Disallow | /braves/search |
Disallow | /brewers/search |
Disallow | /cardinals/search |
Disallow | /cubs/search |
Disallow | /dbacks/search |
Disallow | /dodgers/search |
Disallow | /giants/search |
Disallow | /guardians/search |
Disallow | /mariners/search |
Disallow | /marlins/search |
Disallow | /mets/search |
Disallow | /nationals/search |
Disallow | /orioles/search |
Disallow | /padres/search |
Disallow | /phillies/search |
Disallow | /pirates/search |
Disallow | /rangers/search |
Disallow | /rays/search |
Disallow | /reds/search |
Disallow | /redsox/search |
Disallow | /rockies/search |
Disallow | /royals/search |
Disallow | /tigers/search |
Disallow | /twins/search |
Disallow | /whitesox/search |
Disallow | /yankees/search |
Disallow | /es/search |
Disallow | /es/angels/search |
Disallow | /es/astros/search |
Disallow | /es/athletics/search |
Disallow | /es/bluejays/search |
Disallow | /es/braves/search |
Disallow | /es/brewers/search |
Disallow | /es/cardinals/search |
Disallow | /es/cubs/search |
Disallow | /es/dbacks/search |
Disallow | /es/dodgers/search |
Disallow | /es/giants/search |
Disallow | /es/guardians/search |
Disallow | /es/mariners/search |
Disallow | /es/marlins/search |
Disallow | /es/mets/search |
Disallow | /es/nationals/search |
Disallow | /es/orioles/search |
Disallow | /es/padres/search |
Disallow | /es/phillies/search |
Disallow | /es/pirates/search |
Disallow | /es/rangers/search |
Disallow | /es/rays/search |
Disallow | /es/reds/search |
Disallow | /es/redsox/search |
Disallow | /es/rockies/search |
Disallow | /es/royals/search |
Disallow | /es/tigers/search |
Disallow | /es/twins/search |
Disallow | /es/whitesox/search |
Disallow | /es/yankees/search |
Other Records
Field | Value |
---|---|
sitemap | https://www.mlb.com/sitemaps/48-hr-news.xml.gz |
sitemap | https://www.mlb.com/sitemaps/weekly-news/index.xml.gz |
sitemap | https://www.mlb.com/sitemaps/weekly-video/index.xml.gz |