athletics.mlb.com
robots.txt

Robots Exclusion Standard data for athletics.mlb.com

Resource Scan

Scan Details

Site Domain athletics.mlb.com
Base Domain mlb.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-09T13:32:15+00:00
Next Scan 2025-02-07T13:32:15+00:00

Last Successful Scan

Scanned2024-04-14T12:53:23+00:00
URL http://athletics.mlb.com/robots.txt
Redirect https://www.mlb.com/robots.txt
Redirect Domain www.mlb.com
Redirect Base mlb.com
Domain IPs 34.102.169.19
Redirect IPs 104.18.38.42, 172.64.149.214, 2606:4700:4400::6812:262a, 2606:4700:4400::ac40:95d6
Response IP 104.18.38.42
Found Yes
Hash 70b9f0e0cc82f45953b59b17af9409931bebef41e77e504126092f82a32adb81
SimHash 91dd9b66c41f

Groups

*

Rule Path
Disallow /api/
Disallow /app/
Disallow /embed/
Disallow /en/
Disallow /mlb/
Disallow /share/
Disallow /tokens/
Disallow /tv/
Disallow /web/
Disallow /legacy
Disallow /beta
Disallow /es/legacy
Disallow /es/beta
Disallow /search
Disallow /angels/search
Disallow /astros/search
Disallow /athletics/search
Disallow /bluejays/search
Disallow /braves/search
Disallow /brewers/search
Disallow /cardinals/search
Disallow /cubs/search
Disallow /dbacks/search
Disallow /dodgers/search
Disallow /giants/search
Disallow /guardians/search
Disallow /mariners/search
Disallow /marlins/search
Disallow /mets/search
Disallow /nationals/search
Disallow /orioles/search
Disallow /padres/search
Disallow /phillies/search
Disallow /pirates/search
Disallow /rangers/search
Disallow /rays/search
Disallow /reds/search
Disallow /redsox/search
Disallow /rockies/search
Disallow /royals/search
Disallow /tigers/search
Disallow /twins/search
Disallow /whitesox/search
Disallow /yankees/search
Disallow /es/search
Disallow /es/angels/search
Disallow /es/astros/search
Disallow /es/athletics/search
Disallow /es/bluejays/search
Disallow /es/braves/search
Disallow /es/brewers/search
Disallow /es/cardinals/search
Disallow /es/cubs/search
Disallow /es/dbacks/search
Disallow /es/dodgers/search
Disallow /es/giants/search
Disallow /es/guardians/search
Disallow /es/mariners/search
Disallow /es/marlins/search
Disallow /es/mets/search
Disallow /es/nationals/search
Disallow /es/orioles/search
Disallow /es/padres/search
Disallow /es/phillies/search
Disallow /es/pirates/search
Disallow /es/rangers/search
Disallow /es/rays/search
Disallow /es/reds/search
Disallow /es/redsox/search
Disallow /es/rockies/search
Disallow /es/royals/search
Disallow /es/tigers/search
Disallow /es/twins/search
Disallow /es/whitesox/search
Disallow /es/yankees/search
Disallow /angels/fans/text-messaging-terms-and-conditions
Disallow /astros/fans/text-messaging-terms-and-conditions
Disallow /athletics/fans/text-messaging-terms-and-conditions
Disallow /bluejays/fans/text-messaging-terms-and-conditions
Disallow /braves/fans/text-messaging-terms-and-conditions
Disallow /brewers/fans/text-messaging-terms-and-conditions
Disallow /cardinals/fans/text-messaging-terms-and-conditions
Disallow /cubs/fans/text-messaging-terms-and-conditions
Disallow /dbacks/fans/text-messaging-terms-and-conditions
Disallow /dodgers/fans/text-messaging-terms-and-conditions
Disallow /giants/fans/text-messaging-terms-and-conditions
Disallow /guardians/fans/text-messaging-terms-and-conditions
Disallow /mariners/fans/text-messaging-terms-and-conditions
Disallow /marlins/fans/text-messaging-terms-and-conditions
Disallow /mets/fans/text-messaging-terms-and-conditions
Disallow /nationals/fans/text-messaging-terms-and-conditions
Disallow /orioles/fans/text-messaging-terms-and-conditions
Disallow /padres/fans/text-messaging-terms-and-conditions
Disallow /phillies/fans/text-messaging-terms-and-conditions
Disallow /pirates/fans/text-messaging-terms-and-conditions
Disallow /rangers/fans/text-messaging-terms-and-conditions
Disallow /rays/fans/text-messaging-terms-and-conditions
Disallow /reds/fans/text-messaging-terms-and-conditions
Disallow /redsox/fans/text-messaging-terms-and-conditions
Disallow /rockies/fans/text-messaging-terms-and-conditions
Disallow /royals/fans/text-messaging-terms-and-conditions
Disallow /tigers/fans/text-messaging-terms-and-conditions
Disallow /twins/fans/text-messaging-terms-and-conditions
Disallow /whitesox/fans/text-messaging-terms-and-conditions
Disallow /yankees/fans/text-messaging-terms-and-conditions

Other Records

Field Value
sitemap https://www.mlb.com/sitemaps/48-hr-news.xml.gz
sitemap https://www.mlb.com/sitemaps/weekly-news/index.xml.gz
sitemap https://www.mlb.com/sitemaps/weekly-video/index.xml.gz
sitemap https://www.mlb.com/sitemaps/pages/en/index.xml.gz
sitemap https://www.mlb.com/sitemaps/pages/es/index.xml.gz