winemag.com
robots.txt
Robots Exclusion Standard data for winemag.com
Resource Scan
Scan Details
| Site Domain | winemag.com |
| Base Domain | winemag.com |
| Scan Status | Ok |
| Last Scan | 2024-10-06T09:24:11+00:00 |
| Next Scan | 2024-11-05T09:24:11+00:00 |
Last Scan
| Scanned | 2024-10-06T09:24:11+00:00 |
| URL | https://winemag.com/robots.txt |
| Redirect | https://www.wineenthusiast.com/robots.txt |
| Redirect Domain | www.wineenthusiast.com |
| Redirect Base | wineenthusiast.com |
| Domain IPs | 141.193.213.10, 141.193.213.11 |
| Redirect IPs | 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124 |
| Response IP | 199.232.45.124 |
| Found | Yes |
| Hash | a3fe5966ca4f12b25b52b70f72f3b1ef7e7dd33d2a03b50f2daa3a88323427e2 |
| SimHash | 4915565c8683 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /w/index.php |
| Disallow | /w/*.php |
| Disallow | /w/checkout/ |
| Disallow | /w/app/ |
| Disallow | /w/lib/ |
| Disallow | /w/pkginfo/ |
| Disallow | /w/report/ |
| Disallow | /w/var/ |
| Disallow | /w/catalog/ |
| Disallow | /w/customer/ |
| Disallow | /w/sendfriend/ |
| Disallow | /w/review/ |
| Disallow | /w/*SID%3D |
| Disallow | /w/weltpixel* |
| Disallow | /w/graphql/ |
| Disallow | /w/cpi/ |
| Disallow | /w/banner/ |
| Disallow | /wp-admin |
| Disallow | /search |
| Disallow | /wp-cron.php? |
| Disallow | /?wp-cmd=cron |
| Disallow | /wp-login.php? |
| Disallow | /?s=* |
| Disallow | carousel_articles.json |
| Disallow | carousel_articles.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.wineenthusiast.com/sitemap_index.xml |
| sitemap | https://www.wineenthusiast.com/w/media/sitemap/sitemap.xml |
Comments