fred.com
robots.txt
Robots Exclusion Standard data for fred.com
Resource Scan
Scan Details
Site Domain | fred.com |
Base Domain | fred.com |
Scan Status | Failed |
Failure Reason | Scan timed out. |
Last Scan | 2024-06-19T21:43:08+00:00 |
Next Scan | 2024-07-19T21:43:08+00:00 |
Last Successful Scan
Scanned | 2024-04-28T17:29:24+00:00 |
URL | https://fred.com/robots.txt |
Redirect | https://www.fred.com/robots.txt |
Redirect Domain | www.fred.com |
Redirect Base | fred.com |
Domain IPs | 23.52.171.66, 23.59.168.163, 2600:1413:b000:1b::17d7:70b, 2600:1413:b000:1b::17d7:713 |
Redirect IPs | 23.215.7.11, 23.215.7.19, 2600:1413:b000:1d::17d1:2e98, 2600:1413:b000:1d::17d1:2ea3 |
Response IP | 23.32.29.106 |
Found | Yes |
Hash | 3521ebd12a1705bc8b69f473ba2b9449f7c8d2d49c69ba151e39fb8a30a8ec20 |
SimHash | b8109909c174 |
Groups
*
Rule | Path |
---|---|
Disallow | *Search-ShowAjax* |
Disallow | *Search?q=* |
Disallow | *search?q=* |
Disallow | */Login* |
Disallow | */login* |
Disallow | */AddtoCart* |
Disallow | */cart* |
Disallow | *Wishlist-Show* |
Disallow | */ae/ar_AE* |
Other Records
Field | Value |
---|---|
sitemap | https://www.fred.com/sitemap_index.xml |
sitemap | https://www.fred.com/kr/ko_KR/sitemap_index.xml |
sitemap | https://www.fred.com/jp/ja_JP/sitemap_index.xml |
sitemap | https://www.fred.com/en-int/sitemap_index.xml |
Comments