herbaldrogist.com
robots.txt
Robots Exclusion Standard data for herbaldrogist.com
Resource Scan
Scan Details
| Site Domain | herbaldrogist.com |
| Base Domain | herbaldrogist.com |
| Scan Status | Ok |
| Last Scan | 2025-10-16T08:36:51+00:00 |
| Next Scan | 2025-11-15T08:36:51+00:00 |
Last Scan
| Scanned | 2025-10-16T08:36:51+00:00 |
| URL | https://herbaldrogist.com/robots.txt |
| Redirect | https://www.herbaldrogist.com/robots.txt |
| Redirect Domain | www.herbaldrogist.com |
| Redirect Base | herbaldrogist.com |
| Domain IPs | 104.16.8.49 |
| Redirect IPs | 104.16.8.49 |
| Response IP | 104.16.8.49 |
| Found | Yes |
| Hash | 22f43e3dce34045fac1e97bb42c3e6b280bc102e74c8bc77331905457f959e77 |
| SimHash | 2545fdfe8a13 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /admin |
| Disallow | /nl/account/ |
| Disallow | /de/account/ |
| Disallow | /en/account/ |
| Disallow | /fr/account/ |
| Disallow | /es/account/ |
| Disallow | /tr/account/ |
| Disallow | /nl/cart/ |
| Disallow | /de/cart/ |
| Disallow | /en/cart/ |
| Disallow | /fr/cart/ |
| Disallow | /es/cart/ |
| Disallow | /tr/cart/ |
| Disallow | /nl/compare/ |
| Disallow | /de/compare/ |
| Disallow | /en/compare/ |
| Disallow | /fr/compare/ |
| Disallow | /es/compare/ |
| Disallow | /tr/compare/ |
| Disallow | /nl/checkout/ |
| Disallow | /de/checkout/ |
| Disallow | /en/checkout/ |
| Disallow | /fr/checkout/ |
| Disallow | /es/checkout/ |
| Disallow | /tr/checkout/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.herbaldrogist.com/sitemap.xml |