behrents.com
robots.txt
Robots Exclusion Standard data for behrents.com
Resource Scan
Scan Details
| Site Domain | behrents.com |
| Base Domain | behrents.com |
| Scan Status | Ok |
| Last Scan | 2024-10-23T12:00:31+00:00 |
| Next Scan | 2024-11-22T12:00:31+00:00 |
Last Scan
| Scanned | 2024-10-23T12:00:31+00:00 |
| URL | https://behrents.com/robots.txt |
| Domain IPs | 216.59.42.212 |
| Response IP | 216.59.42.212 |
| Found | Yes |
| Hash | 8268ef8817ef1c3ccaca8ff802d6e7c258770d9d96b110fe58faa1a9ca6d1e83 |
| SimHash | 6503dde5bdd1 |
Groups
*
*
| Rule | Path |
|---|---|
| Disallow | /aspnet_client/ |
| Disallow | /httperrors/ |
| Disallow | /HTTPErrors/ |
| Disallow | /create_login_and_account.asp* |
| Disallow | /showcart.asp* |
| Disallow | /account.asp* |
| Disallow | /payment.asp* |
| Disallow | /add_product_to_favorites.asp* |
| Disallow | /ae_sf_review_comment.asp* |
| Disallow | /add_to_cart.asp* |
| Disallow | /search.asp* |
| Disallow | /i_i_add_product_to_favorites.asp* |
| Disallow | /i_i_ae_sf_review_comment.asp* |
| Disallow | /i_i_add_to_cart.asp* |
| Disallow | /i_i_linkset.asp* |
| Disallow | /aspnet_client/ |
| Disallow | /Templates/ |
| Disallow | /httperrors/ |
| Disallow | /HTTPErrors/ |
| Disallow | /temp/ |
| Disallow | /test/ |
| Disallow | /pc_combined_results.asp |
| Disallow | /i_i_add_product_to_favorites.asp |
| Disallow | /page.asp |
| Disallow | /i_i_priceBreakPopup.asp |
| Disallow | /prod_inv_watch_add.asp |
| Disallow | /pc_product_detail.asp |
| Disallow | /security_logon.asp |
| Disallow | /i_i_email_friend_request.asp |
| Disallow | /order_form.asp |
| Disallow | /create_login_select_acct.asp |
| Disallow | /order_form.asp |
| Disallow | /request_password_reset.asp |
| Disallow | /request_username.asp |
| Disallow | /showcart.asp |
| Disallow | /parts/ae_sf_review_comment.asp? |
| Disallow | /parts/i_i_prod_inv_watch_add.asp? |
| Disallow | /parts/security_logon.asp? |
| Disallow | /parts/i_i_email_friend_request.asp? |
| Disallow | /parts/i_i_add_product_to_favorites.asp? |
| Disallow | /parts/i_i_email_friend_request.asp? |
| Disallow | /parts/i_i_priceBreakPopup.asp? |
googlebot
| Rule | Path |
|---|---|
| Disallow | /aspnet_client/ |
| Disallow | /httperrors/ |
| Disallow | /HTTPErrors/ |
| Disallow | /create_login_and_account.asp* |
| Disallow | /showcart.asp* |
| Disallow | /account.asp* |
| Disallow | /payment.asp* |
| Disallow | /add_product_to_favorites.asp* |
| Disallow | /ae_sf_review_comment.asp* |
| Disallow | /add_to_cart.asp* |
| Disallow | /search.asp* |
| Disallow | /i_i_add_product_to_favorites.asp* |
| Disallow | /i_i_ae_sf_review_comment.asp* |
| Disallow | /i_i_add_to_cart.asp* |
| Disallow | /i_i_linkset.asp* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
googlebot-image
| Rule | Path |
|---|---|
| Disallow | /aspnet_client/ |
| Disallow | /httperrors/ |
| Disallow | /HTTPErrors/ |
| Disallow | /create_login_and_account.asp* |
| Disallow | /showcart.asp* |
| Disallow | /account.asp* |
| Disallow | /payment.asp* |
bingbot
| Rule | Path |
|---|---|
| Disallow | /aspnet_client/ |
| Disallow | /httperrors/ |
| Disallow | /HTTPErrors/ |
| Disallow | /create_login_and_account.asp* |
| Disallow | /showcart.asp* |
| Disallow | /account.asp* |
| Disallow | /payment.asp* |
| Disallow | /add_product_to_favorites.asp* |
| Disallow | /ae_sf_review_comment.asp* |
| Disallow | /add_to_cart.asp* |
| Disallow | /search.asp* |
| Disallow | /i_i_add_product_to_favorites.asp* |
| Disallow | /i_i_ae_sf_review_comment.asp* |
| Disallow | /i_i_add_to_cart.asp* |
| Disallow | /i_i_linkset.asp* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 45 |
slurp
| Rule | Path |
|---|---|
| Disallow | /aspnet_client/ |
| Disallow | /httperrors/ |
| Disallow | /HTTPErrors/ |
| Disallow | /create_login_and_account.asp* |
| Disallow | /showcart.asp* |
| Disallow | /account.asp* |
| Disallow | /payment.asp* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 15 |
msnbot
| Rule | Path |
|---|---|
| Disallow | /aspnet_client/ |
| Disallow | /httperrors/ |
| Disallow | /HTTPErrors/ |
| Disallow | /create_login_and_account.asp* |
| Disallow | /showcart.asp* |
| Disallow | /account.asp* |
| Disallow | /payment.asp* |
| Disallow | /add_product_to_favorites.asp* |
| Disallow | /ae_sf_review_comment.asp* |
| Disallow | /add_to_cart.asp* |
| Disallow | /search.asp* |
| Disallow | /i_i_add_product_to_favorites.asp* |
| Disallow | /i_i_ae_sf_review_comment.asp* |
| Disallow | /i_i_add_to_cart.asp* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 45 |
duckduckbot
| Rule | Path |
|---|---|
| Disallow | /aspnet_client/ |
| Disallow | /httperrors/ |
| Disallow | /HTTPErrors/ |
| Disallow | /create_login_and_account.asp* |
| Disallow | /showcart.asp* |
| Disallow | /account.asp* |
| Disallow | /payment.asp* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
rogerbot
| Rule | Path |
|---|---|
| Disallow | /aspnet_client/ |
| Disallow | /httperrors/ |
| Disallow | /HTTPErrors/ |
| Disallow | /create_login_and_account.asp* |
| Disallow | /showcart.asp* |
| Disallow | /account.asp* |
| Disallow | /payment.asp* |
| Disallow | /add_product_to_favorites.asp* |
| Disallow | /ae_sf_review_comment.asp* |
| Disallow | /add_to_cart.asp* |
| Disallow | /search.asp* |
| Disallow | /i_i_add_product_to_favorites.asp* |
| Disallow | /i_i_ae_sf_review_comment.asp* |
| Disallow | /i_i_add_to_cart.asp* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 45 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://behrents.com/sitemap_index.asp |
Comments