dherbs.com
robots.txt
Robots Exclusion Standard data for dherbs.com
Resource Scan
Scan Details
| Site Domain | dherbs.com |
| Base Domain | dherbs.com |
| Scan Status | Ok |
| Last Scan | 2024-06-18T11:39:23+00:00 |
| Next Scan | 2024-07-18T11:39:23+00:00 |
Last Scan
| Scanned | 2024-06-18T11:39:23+00:00 |
| URL | https://dherbs.com/robots.txt |
| Redirect | https://www.dherbs.com/robots.txt |
| Redirect Domain | www.dherbs.com |
| Redirect Base | dherbs.com |
| Domain IPs | 54.219.128.192 |
| Redirect IPs | 54.219.128.192 |
| Response IP | 54.219.128.192 |
| Found | Yes |
| Hash | 52fd7894cad145a436ca4d0d28551d78c3899d897fd9a55e7d94e8efbbf36937 |
| SimHash | b05ad067c2e5 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /bmz_cache/ |
| Disallow | /cache/ |
| Disallow | /media/ |
| Disallow | /pics/ |
| Disallow | /slot/ |
| Disallow | /pub/ |
| Disallow | /temp/ |
| Disallow | /libs/ |
| Disallow | /pdf/ |
| Allow | /action/?task=sitemapgen* |
| Disallow | /action/ |
| Disallow | /header/ |
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /store/index.php?main_page=product_reviews_ajax* |
| Disallow | /store/index.php?main_page=product_faq_ajax* |
| Disallow | /store/index.php?main_page=fetch* |
| Disallow | /store/index.php?main_page=popup_whitelist_dherbs* |
| Disallow | /store/app_* |
| Disallow | /store/api* |
| Disallow | /store/newsletter_signup.html?h=* |
| Disallow | /store/action.php* |
| Disallow | /store/popup_whitelist_dherbs* |
| Disallow | /store/shipstation* |
| Disallow | /store/dhtrk.php* |
| Disallow | /store/uploads/* |
| Disallow | /cache/ |
| Disallow | /test-radio/* |
| Disallow | /flash/* |
| Disallow | /raf/* |
| Disallow | /support/* |
| Disallow | /did-the-fda-mailed-dherbs-com-a-warning-letter/* |
| Disallow | /ajax_* |
| Disallow | /wp-api-dherbs.php* |
| Disallow | /featured/wp-api-dherbs.php* |
| Disallow | /dtx-troubleshooting* |
| Disallow | /?noCache* |
| Disallow | *?vid=* |
| Disallow | /*noCache* |
| Disallow | /?nocache* |
| Disallow | /*nocache* |
| Disallow | /store/?noCache* |
| Disallow | /store/?nocache* |
| Disallow | /store/ajax_* |
| Disallow | /store/*coupon%3D* |
| Disallow | /store/*product_reviews_write* |
| Disallow | /store/*login* |
| Disallow | /store/*account* |
| Disallow | /store/*wishlist* |
| Disallow | /store/*auto_delivery* |
| Disallow | /store/*distributor_information* |
| Disallow | /store/app_customer* |
| Disallow | /store/product_finder_local* |
| Disallow | /store/banner_preview* |
| Disallow | /content/commonFooterCode* |
| Disallow | /search-x* |
| Disallow | /search* |
| Disallow | /store/index.php?keyword* |
| Disallow | /forums/ |
| Disallow | /forums/index.php?action=profile* |
| Disallow | /forums/*action |
| Disallow | /forums/*board%3D*wap |
| Disallow | /forums/*board%3D*imode |
| Disallow | /forums/*topic%3D*wap |
| Disallow | /forums/*topic%3D*imode |
| Disallow | /forums/*%3B |
| Disallow | /forums/*PHPSESSID |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.dherbs.com/sitemap.xml |
| sitemap | https://www.dherbs.com/store/sitemap.xml |
| sitemap | https://www.dherbs.com/store/rss_specials.php |
| sitemap | https://www.dherbs.com/store/sitemapgen.php?type=htpages |
| sitemap | https://www.dherbs.com/store/sitemapgen.php?type=fbcomments |
| sitemap | https://www.dherbs.com/store/sitemapgen.php?type=ingredient_search |
| sitemap | https://www.dherbs.com/store/sitemapgen.php?type=sitemap_faqs |
| sitemap | https://www.dherbs.com/store/sitemapgen.php?type=herbalist_pages |
| sitemap | https://www.dherbs.com/store/sitemapgen.php?type=herbalist_item |
| sitemap | https://www.dherbs.com/action/?task=sitemapgen&type=newsletter_archive_xml |