luxsale.com
robots.txt

Robots Exclusion Standard data for luxsale.com

Resource Scan

Scan Details

Site Domain luxsale.com
Base Domain luxsale.com
Scan Status Ok
Last Scan2024-10-17T21:13:38+00:00
Next Scan 2024-11-16T21:13:38+00:00

Last Scan

Scanned2024-10-17T21:13:38+00:00
URL https://luxsale.com/robots.txt
Redirect https://www.larkbury.com/robots.txt
Redirect Domain www.larkbury.com
Redirect Base larkbury.com
Domain IPs 13.225.4.102, 13.225.4.33, 13.225.4.4, 13.225.4.83
Redirect IPs 87.76.19.7
Response IP 87.76.19.7
Found Yes
Hash 0b4451d0a57a80b89f2454247f27649f0269699d3b3316d88bb66cea7c6a5788
SimHash 2904f47a4abb

Groups

*

Rule Path
Disallow /repo/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalog/seo_sitemap/
Disallow /catalog/
Disallow /catalogsearch/
Disallow /mfproductsearch/
Disallow /search/
Disallow /checkout/
Disallow /checkout/cart/
Disallow /onestepcheckout/
Disallow /control/
Disallow /customer/
Disallow /customer/account/
Disallow /customer/account/login/
Disallow /customer/account/login/referer/
Disallow /customer/account/forgotpassword/
Disallow /customize/
Disallow /sendfriend/
Disallow /ajaxcart/
Disallow /ajax/
Disallow /quickview/
Disallow /productalert/
Disallow /mfcmsdr/
Disallow /sales/guest/form/
Disallow /sales/guest/form/
Disallow /review/
Disallow /downloadable/
Disallow /pslogin
Disallow /subscription
Disallow /newsletter
Disallow /push_notification
Disallow /var/
Disallow /pub/sap/
Disallow /sap/
Disallow /index.php
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /STATUS.txt
Disallow /get.php
Disallow /app/
Disallow /lib/
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /privacy-policy-cookie-restriction-mode
Disallow /no-route
Disallow /enable-cookies
Disallow /home

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

sogou spider2

Rule Path
Disallow /

sogou blog

Rule Path
Disallow /

sogou news spider

Rule Path
Disallow /

sogou orion spider

Rule Path
Disallow /

bingpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefssiteaudit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.larkbury.com/media/sitemap/sitemap.xml

Comments

  • Files
  • CMS Pages

Warnings

  • 2 invalid lines.