bexley.com
robots.txt

Robots Exclusion Standard data for bexley.com

Resource Scan

Scan Details

Site Domain bexley.com
Base Domain bexley.com
Scan Status Ok
Last Scan2025-05-25T09:45:37+00:00
Next Scan 2025-06-08T09:45:37+00:00

Last Scan

Scanned2025-05-25T09:45:37+00:00
URL https://bexley.com/robots.txt
Redirect https://www.bexley.com/robots.txt
Redirect Domain www.bexley.com
Redirect Base bexley.com
Domain IPs 128.204.208.220
Redirect IPs 18.161.97.12, 18.161.97.47, 18.161.97.7, 18.161.97.86
Response IP 52.85.49.122
Found Yes
Hash a4b7d15e40760d4f4766212dbb03eaacf8a9ebbe684c5dce9abc6364639a852e
SimHash 2b26fa47c13a

Groups

*

Rule Path
Disallow /index.php/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /wishlist/
Disallow /admin/
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /sendfriend/
Disallow /enable-cookies/
Disallow /LICENSE.txt
Disallow /LICENSE.html
Disallow /skin/
Disallow /js/
Disallow /directory/
Disallow /customer/
Disallow /customer/account/
Disallow /customer/account/login/
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /pub/
Disallow /tag/
Disallow /review/
Disallow /*?refSrc=
Disallow /*?nosto=
Disallow /*?selecteur=
Disallow /*?&refSrc=
Disallow /*?&nosto=
Disallow /*?&selecteur=
Disallow /*?order=
Disallow /*%26order%3D
Disallow /*?dir=
Disallow /*%26dir%3D
Disallow /*?limit=
Disallow /*%26limit%3D
Disallow /*?mode=
Disallow /*%26mode%3D
Disallow /*?price=
Disallow /*%26price%3D
Disallow /*?product_list_order=
Disallow /*%26product_list_order%3D
Disallow /*SID%3D
Disallow /aw_rma/
Disallow /*.php$
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /amasty_promo/popup/reload/
Disallow /sales/
Disallow /physicalsales/
Disallow /bexley_rma/
Disallow /newsletter/

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

mj12bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

exabot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

Comments

  • Default robots.txt for bexley.com