Log In Sign Up

fashionesta.com
robots.txt

Robots Exclusion Standard data for fashionesta.com

Resource Scan

Scan Details

Site Domain fashionesta.com
Base Domain fashionesta.com
Scan Status Ok
Last Scan2024-10-22T14:25:53+00:00
Next Scan 2024-11-21T14:25:53+00:00

Last Scan

Scanned2024-10-22T14:25:53+00:00
URL https://fashionesta.com/robots.txt
Redirect https://www.fashionesta.com/robots.txt
Redirect Domain www.fashionesta.com
Redirect Base fashionesta.com
Domain IPs 151.101.130.132, 151.101.194.132, 151.101.2.132, 151.101.66.132
Redirect IPs 151.101.130.132, 151.101.194.132, 151.101.2.132, 151.101.66.132
Response IP 199.232.46.132
Found Yes
Hash 3a4d5556528807f53952450f4095c9d97059efd98dff65079b2bf84e4440042c
SimHash 3916af518771

Groups

*

Rule Path Comment
Allow /catalog/seo_sitemap/category/ -
Allow /media/catalog/ -
Allow /*?utm* -
Disallow /404/ -
Disallow /app/ -
Disallow /cgi-bin/ -
Disallow /downloader/ -
Disallow /errors/ -
Disallow /includes/ -
Disallow /js/ -
Disallow /lib/ -
Disallow /magento/ -
Disallow /media/captcha/ -
Disallow /media/customer/ -
Disallow /media/dhl/ -
Disallow /media/downloadable/ -
Disallow /media/import/ -
Disallow /media/pdf/ -
Disallow /media/sales/ -
Disallow /media/tmp/ -
Disallow /media/wysiwyg/ -
Disallow /media/xmlconnect/ -
Disallow /pkginfo/ -
Disallow /report/ -
Disallow /scripts/ -
Disallow /shell/ -
Disallow /stats/ -
Disallow /var/ -
Disallow /index.php/ -
Disallow /catalog/product_compare/ -
Disallow /catalog/category/view/ -
Disallow /catalog/product/view/ -
Disallow /catalog/product/gallery/ -
Disallow /catalogsearch/ -
Disallow /checkout/ -
Disallow /control/ -
Disallow /contacts/ -
Disallow /customer/ -
Disallow /customize/ -
Disallow /newsletter/ -
Disallow /poll/ -
Disallow /review/ -
Disallow /sendfriend/ -
Disallow /tag/ -
Disallow /wishlist/ -
Disallow /cron.php -
Disallow /cron.sh -
Disallow /error_log -
Disallow /install.php -
Disallow /LICENSE.html -
Disallow /LICENSE.txt -
Disallow /LICENSE_AFL.txt -
Disallow /STATUS.txt -
Disallow /get.php Magento 1.5+
Disallow /*.php$ -
Disallow /*?SID= -
Disallow /rss* -
Disallow /*PHPSESSID -
Disallow /*limit%3D* -
Disallow /*order%3D* -
Disallow /*dir%3D* -
Disallow /*price%3D* -
Disallow /*multiple%3D* -
Disallow /*? -
Allow /js/ -
Allow /media/catalog/cache/ -
Allow /media/catalog/category/ -
Allow /media/catalog/product/ -
Allow /media/css/ -
Allow /media/js/ -
Allow /skin/frontend/base/default/css/ -
Allow /skin/frontend/base/default/images/ -
Allow /skin/frontend/base/default/js/ -
Allow /skin/frontend/default/blank/css/ -
Allow /skin/frontend/default/blank/images/ -
Allow /skin/frontend/default/blue/css/ -
Allow /skin/frontend/default/blue/images/ -
Allow /skin/frontend/default/default/css/ -
Allow /skin/frontend/default/default/images/ -
Allow /skin/frontend/default/iphone/css/ -
Allow /skin/frontend/default/iphone/images/ -
Allow /skin/frontend/default/iphone/js/ -
Allow /skin/frontend/default/french/images/ -
Allow /skin/frontend/default/german/images/ -
Allow /skin/frontend/default/modern/css/ -
Allow /skin/frontend/default/modern/images/ -
Allow /skin/frontend/rwd/css/ -
Allow /skin/frontend/rwd/images/ -
Allow /skin/frontend/rwd/js/ -
Allow /skin/frontend/rwd/scss/ -
Disallow /women/ -
Disallow /men/ -
Disallow /sale/ -
Disallow /hommes/ -
Disallow /femmes/ -
Disallow /promo -
Disallow /retoure -
Disallow /ueber-uns -
Disallow /privacy-policy-cookie-restriction-mode -
Disallow /agb -
Disallow /impressum -
Disallow /versand -
Disallow /zahlarten -
Disallow /faq -
Disallow /newsletter -
Disallow /contacts -
Disallow /shipping -
Disallow /return-policy -
Disallow /about-us -
Disallow /imprint -
Disallow /return-policy -
Disallow /payment-methods -
Disallow /a-propos-de-nous -
Disallow /politique-de-confidentialite -
Disallow /conditions-generales -
Disallow /empreinte -
Disallow /retours -
Disallow /paiements -
Disallow */fashionesta/product/sizetable/product -
Disallow */fashionesta/product/question/product -
Disallow */fashionesta/product/tellafriend/product -
Disallow */fashionesta/product/expressCheckout -
Allow /skin/frontend/bootstrap/fashionesta/css/ -
Allow /skin/frontend/bootstrap/fashionesta/images/ -
Allow /skin/frontend/bootstrap/fashionesta/js/ -

Other Records

Field Value
crawl-delay 25

mj12bot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.fashionesta.com/sitemap/sitemap_de_de.xml
sitemap https://www.fashionesta.com/sitemap/sitemap_en_gb.xml
sitemap https://www.fashionesta.com/sitemap/sitemap_fr_fr.xml
sitemap https://www.fashionesta.com/sitemap/sitemap_non_eu_en.xml
sitemap https://www.fashionesta.com/sitemap/sitemap_en_eu.xml
sitemap https://www.fashionesta.com/sitemap/sitemap_de_eu.xml
sitemap https://www.fashionesta.com/sitemap/sitemap_fr_eu.xml
sitemap https://www.fashionesta.com/sitemap/sitemap_en_int.xml
sitemap https://www.fashionesta.com/sitemap/sitemap_de_int.xml
sitemap https://www.fashionesta.com/sitemap/sitemap_fr_int.xml
Back to top

Comments

  • # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
  • Experius generic robots.txt v1.0
  • Always check the contents of this files
  • and modify it to make it customer specific
  • # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
  • ----------------------------------------------------
  • Website Sitemaps
  • IMPORTANT: Modify these according to customer's sitemaps
  • ----------------------------------------------------
  • ----------------------------------------------------
  • Crawlers Setup
  • ----------------------------------------------------
  • ----------------------------------------------------
  • Allowable Index
  • ----------------------------------------------------
  • ----------------------------------------------------
  • Directories
  • ----------------------------------------------------
  • ----------------------------------------------------
  • Paths (clean URLs)
  • ----------------------------------------------------
  • ----------------------------------------------------
  • Files
  • ----------------------------------------------------
  • ----------------------------------------------------
  • Paths (no clean URLs)
  • ----------------------------------------------------
  • ----------------------------------------------------
  • Paths used by Magento's sorting or layered navigation
  • ----------------------------------------------------
  • ----------------------------------------------------
  • Allow css, js and images for Google
  • ----------------------------------------------------
  • ----------------------------------------------------
  • Customer specific
  • ----------------------------------------------------
  • Disallow sizetable, ask a question, tellafriend
  • ----------------------------------------------------
  • Custom themes
  • IMPORTANT: Modify these paths according to customer's theme
  • ----------------------------------------------------
  • ----------------------------------------------------
  • Block bad crawlers
  • ----------------------------------------------------
  • third party search engine bot
  • Web-crawler by the company 'WiseGuys' to gather unstructured webdata
  • Chinese Search Engine
  • French Search Engine
  • Russian Search Engine
  • Security issue crawler (not needed)
Back to top