thefreshmarket.com
robots.txt
Robots Exclusion Standard data for thefreshmarket.com
Resource Scan
Scan Details
| Site Domain | thefreshmarket.com |
| Base Domain | thefreshmarket.com |
| Scan Status | Ok |
| Last Scan | 2024-10-26T00:12:04+00:00 |
| Next Scan | 2024-11-25T00:12:04+00:00 |
Last Scan
| Scanned | 2024-10-26T00:12:04+00:00 |
| URL | https://thefreshmarket.com/robots.txt |
| Redirect | https://www.thefreshmarket.com/robots.txt |
| Redirect Domain | www.thefreshmarket.com |
| Redirect Base | thefreshmarket.com |
| Domain IPs | 3.165.82.116, 3.165.82.16, 3.165.82.52, 3.165.82.69 |
| Redirect IPs | 2600:9000:2079:400:d:d7cb:b200:93a1, 2600:9000:2079:5400:d:d7cb:b200:93a1, 2600:9000:2079:600:d:d7cb:b200:93a1, 2600:9000:2079:6e00:d:d7cb:b200:93a1, 2600:9000:2079:ac00:d:d7cb:b200:93a1, 2600:9000:2079:b200:d:d7cb:b200:93a1, 2600:9000:2079:be00:d:d7cb:b200:93a1, 2600:9000:2079:ea00:d:d7cb:b200:93a1, 3.165.102.3, 3.165.102.35, 3.165.102.64, 3.165.102.7 |
| Response IP | 3.165.102.3 |
| Found | Yes |
| Hash | b1455f32db349c3444e5d8f141b0e7dd14f2c211870c9f1c9e2d33b268ad29dc |
| SimHash | 94afc6486a19 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /account/ |
| Disallow | /club-hub/ |
| Disallow | /coupons-and-offers/ |
| Disallow | /dashboard/ |
| Disallow | /my-wallet/ |
| Disallow | /rewards/ |