Log In Sign Up

cincojotas.es
robots.txt

Robots Exclusion Standard data for cincojotas.es

Resource Scan

Scan Details

Site Domain cincojotas.es
Base Domain cincojotas.es
Scan Status Ok
Last Scan2024-06-06T01:12:54+00:00
Next Scan 2024-07-06T01:12:54+00:00

Last Scan

Scanned2024-06-06T01:12:54+00:00
URL https://cincojotas.es/robots.txt
Redirect https://www.cincojotas.es/robots.txt
Redirect Domain www.cincojotas.es
Redirect Base cincojotas.es
Domain IPs 103.23.61.9
Redirect IPs 23.54.155.79, 23.54.155.80, 2600:1413:b000:13::b857:c193, 2600:1413:b000:13::b857:c1a0
Response IP 184.27.123.33
Found Yes
Hash ac59db8b12cab47c3ed57362a2cb0944b529da1850a24f33c55c5bde0e89e64c
SimHash 2521f315c7f2

Groups

*

Rule Path
Disallow /CVS
Disallow /*.svn$
Disallow /*.idea$
Disallow /*.sql$
Disallow /*.tgz$
Disallow /admin/
Disallow /app/
Disallow /downloader/
Disallow /errors/
Disallow /includes/
Disallow /lib/
Disallow /pkginfo/
Disallow /shell/
Disallow /var/
Disallow /api.php
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /get.php
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /README.txt
Disallow /RELEASE_NOTES.txt
Disallow /*?dir*
Disallow /*?dir=desc
Disallow /*?dir=asc
Disallow /*?limit=all
Disallow /*?mode*
Disallow /*?min*
Disallow /*?max*
Disallow /*?q*
Disallow /*?cat*
Disallow /*?manufacturer_list*
Disallow /*?tx_indexedsearch
Disallow /*?p*
Disallow /*?q*
Disallow /*?id*
Disallow /*?order*
Disallow /*?cont*
Disallow /index.php/
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /customer/
Disallow /customer/account/
Disallow /customer/account/login/
Disallow /wishlist/
Disallow /sendfriend/
Disallow /catalogsearch/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /cgi-bin/
Disallow /cleanup.php
Disallow /apc.php
Disallow /memcache.php
Disallow /phpinfo.php

Other Records

Field Value
crawl-delay 30
Back to top

Other Records

Field Value
sitemap https://www.cincojotas.com/sitemap.xml
Back to top

Comments

  • robots.txt for Magento Community and Enterprise
  • GENERAL SETTINGS
  • Enables robots.txt rules for all crawlers
  • Crawl-delay parameter - the number of seconds you want to wait between successful requests to the same server.
  • Set a crawl rate, if your server's traffic problems. Please note that Google ignore crawl-delay setting in Robots.txt.
  • You can set up this in Google Webmaster tool
  • URL to your sitemap file in Magento
  • DEV/PRE SETTINGS
  • Do not allow indexing files and folders that are required during development: CVS, SVN directory and dump files
  • GENERAL SETTINGS FOR MAGENTO
  • Do not index the page Magento admin
  • Do not index the general technical Magento directory
  • Do not index the shared files Magento
  • MAGENTO SEO IMPROVEMENT
  • Do not index the page subcategories that are sorted or filtered.
  • Audit SEO
  • Do not index the second copy of the home page (example.com/index.php/). Un-comment only if you have activated Magento SEO URLs.
  • Do not index the link from the session ID
  • disallow:/*.php$
  • Do not index the page checkout and user account
  • Do not index the search page and CEO, non-optimized link categories
  • SERVER SETTINGS
  • Do not index the general technical directories and files on a server
  • IMAGE INDEXING SETTINGS
  • Optional: If you do not want to Google and Bing to index your images
  • user-agent: Googlebot-Image
  • disallow:/
  • user-agent: msnbot-media
  • disallow:/
Back to top

Warnings

  • 1 invalid line.
Back to top