winesearcher.com
robots.txt
Robots Exclusion Standard data for winesearcher.com
Resource Scan
Scan Details
Site Domain | winesearcher.com |
Base Domain | winesearcher.com |
Scan Status | Ok |
Last Scan | 2024-09-20T08:18:46+00:00 |
Next Scan | 2024-09-27T08:18:46+00:00 |
Last Scan
Scanned | 2024-09-20T08:18:46+00:00 |
URL | https://winesearcher.com/robots.txt |
Redirect | https://www.wine-searcher.com/robots.txt |
Redirect Domain | www.wine-searcher.com |
Redirect Base | wine-searcher.com |
Domain IPs | 151.101.130.137, 151.101.194.137, 151.101.2.137, 151.101.66.137 |
Redirect IPs | 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91 |
Response IP | 151.101.193.91 |
Found | Yes |
Hash | 642911d20a26fb4aa0406126ab0a4b2664d6c29968c71956dd2e3e2e4a23a62e |
SimHash | 32bd65409d51 |
Groups
*
Rule | Path |
---|---|
Disallow | *wine_id |
Disallow | *Xprice_set |
Disallow | *Xbottle_size |
Disallow | *Xcurrencycode |
Disallow | *Xpurl |
Disallow | *Xsort_order |
Disallow | /redirect* |
Disallow | /app-html/ |
Disallow | /json-country/ |
Disallow | /json-region/ |
Disallow | /json-recommend/ |
Disallow | /widgets/ |
Disallow | /securimage/ |
Disallow | /scratch/ |
Disallow | /portal/ |
Disallow | /site-search* |
Disallow | /wine-notes-search* |
Disallow | /sign-in* |
Disallow | /prof/index* |
Disallow | *pro_redirect* |