Log In Sign Up

drabel.com
robots.txt

Robots Exclusion Standard data for drabel.com

Resource Scan

Scan Details

Site Domain drabel.com
Base Domain drabel.com
Scan Status Ok
Last Scan2024-11-16T21:30:35+00:00
Next Scan 2024-11-23T21:30:35+00:00

Last Scan

Scanned2024-11-16T21:30:35+00:00
URL https://drabel.com/robots.txt
Domain IPs 208.90.22.215
Response IP 208.90.22.215
Found Yes
Hash adbee0a31470d3feefa93092fc6ed3d9192230c292d5f248930f56a9b4d57737
SimHash 261adde1fed6

Groups

pi-monster

Rule Path
Disallow /

pimonster

Rule Path
Disallow /

pimonster

Rule Path
Disallow /
Disallow /

psbot

Rule Path
Disallow /

pi-monster

Rule Path
Disallow /

pricepi

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

shopzilla

Rule Path
Disallow /

nakhodka

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

baiduspider-favo

Rule Path
Disallow /

baiduspider-cpro

Rule Path
Disallow /

baiduspider-ads

Rule Path
Disallow /

baidu

Rule Path
Disallow /

yrspider

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

jikespider \"mozilla/5.0"

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

corp.bahnhof.se

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

dataforseobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 25
crawl-delay 25

mediapartners-google

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap https://www.drabel.com/sitemapindex.xml
sitemap https://www.drabel.com/sitemapproducts.xml
sitemap https://www.drabel.com/sitemap_mobileindex.xml
Back to top

Comments

  • robots.txt for https://www.drabel.com/
  • Email: Dennis@drabel.com
  • Block pricepi
  • User-agent: AhrefsBot
  • Crawl-Delay: 60
  • User-agent: *
  • Allow: /
  • Disallow: /~
  • Disallow: /account.php
  • Disallow: /account_edit.php
  • Disallow: /account_edit_process.php
  • Disallow: /account_history.php
  • Disallow: /account_history_info.php
  • Disallow: /account_notifications.php
  • Disallow: /account_password.php
  • Disallow: /add_checkout_success.php
  • Disallow: /address_book.php
  • Disallow: /address_book_process.php
  • Disallow: /addtocart.php
  • Disallow: /advanced_search.php
  • Disallow: /advanced_search_result.php
  • Disallow: /cgi-bin/
  • Disallow: /checkout_confirmation.php
  • Disallow: /checkout_payment.php
  • Disallow: /checkout_payment_address.php
  • Disallow: /checkout_paypalipn.php
  • Disallow: /checkout_process.php
  • Disallow: /checkout_shipping.php
  • Disallow: /checkout_shipping_address.php
  • Disallow: /checkout_success.php
  • Disallow: /cookie_usage.php
  • Disallow: /create_account.php
  • Disallow: /create_account_process.php
  • Disallow: /create_account_success.php
  • Disallow: /download.php
  • Disallow: /download/
  • Disallow: /free-downloads.php
  • Disallow: /includes/
  • Disallow: /info_shopping_cart.php
  • Disallow: /login.php
  • Disallow: /logoff.php
  • Disallow: /password_forgotten.php
  • Disallow: /popup_coupon_help.php
  • Disallow: /popup_image.php
  • Disallow: /popup_search_help.php
  • Disallow: /product_notifications.php
  • Disallow: /pub/
  • Disallow: /redirect.php
  • Disallow: /reunion/
  • Disallow: /shipping_estimator_popup.php
  • Disallow: /shopping_cart.php
  • Disallow: /shopping_cart_help.php
  • Disallow: /stats_products.php:
  • Disallow: /tell_a_friend.php
  • Disallow: /temp/
  • To ensure the 'googlebot' is not being blocked, please add the following two lines of text to the end of your robots.txt file:
  • For Google sitemap
Back to top

Warnings

  • 13 invalid lines.
  • `user-name` is not a known field.
Back to top