retif.eu
robots.txt

Robots Exclusion Standard data for retif.eu

Resource Scan

Scan Details

Site Domain retif.eu
Base Domain retif.eu
Scan Status Ok
Last Scan2024-09-16T04:22:49+00:00
Next Scan 2024-10-16T04:22:49+00:00

Last Scan

Scanned2024-09-16T04:22:49+00:00
URL https://retif.eu/robots.txt
Redirect https://www.retif.eu/robots.txt
Redirect Domain www.retif.eu
Redirect Base retif.eu
Domain IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Redirect IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Response IP 199.232.45.124
Found Yes
Hash 2ce4397d6f385780f44a87611d106bd7ed53fd557ec533918ba12e83ff48851e
SimHash ce04bcd103b1

Groups

*

Rule Path
Disallow /index.php/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalog/product/gallery/
Disallow /catalogsearch/
Disallow /checkout/
Disallow /control/
Disallow /contacts/
Disallow /customer/
Disallow /customize/
Disallow /newsletter/
Disallow /poll/
Disallow /review/
Disallow /sendfriend/
Disallow /tag/
Disallow /wishlist/
Disallow /*product_list_order*
Disallow /*product_list_limit%3D*
Disallow /*filters%3D*

*

Rule Path Comment
Allow /*?p= -
Allow /index.php/blog/ -
Allow /catalog/seo_sitemap/category/ -
Allow /media/catalog/ -
Disallow /404/ -
Disallow /app/ -
Disallow /cgi-bin/ -
Disallow /downloader/ -
Disallow /errors/ -
Disallow /includes/ -
Disallow /lib/ -
Disallow /magento/ -
Disallow /media/captcha/ -
Disallow /media/customer/ -
Disallow /media/dhl/ -
Disallow /media/downloadable/ -
Disallow /media/import/ -
Disallow /media/pdf/ -
Disallow /media/sales/ -
Disallow /media/tmp/ -
Disallow /media/wysiwyg/ -
Disallow /media/xmlconnect/ -
Disallow /pkginfo/ -
Disallow /report/ -
Disallow /scripts/ -
Disallow /shell/ -
Disallow /stats/ -
Disallow /var/ -
Disallow /index.php/ -
Disallow /catalog/product_compare/ -
Disallow /catalog/category/view/ -
Disallow /catalog/product/view/ -
Disallow /catalog/product/gallery/ -
Disallow /catalogsearch/ -
Disallow /checkout/ -
Disallow /control/ -
Disallow /contacts/ -
Disallow /customer/ -
Disallow /customize/ -
Disallow /newsletter/ -
Disallow /poll/ -
Disallow /review/ -
Disallow /sendfriend/ -
Disallow /tag/ -
Disallow /wishlist/ -
Disallow /cron.php -
Disallow /cron.sh -
Disallow /error_log -
Disallow /install.php -
Disallow /LICENSE.html -
Disallow /LICENSE.txt -
Disallow /LICENSE_AFL.txt -
Disallow /STATUS.txt -
Disallow /get.php Magento 1.5+
Disallow /*?dir=* -
Disallow /*%26dir%3D* -
Disallow /*.php$ -
Disallow /*?SID= -
Disallow /?___from_store= -
Disallow /*___from_store%3D -
Disallow /rss* -
Disallow /*PHPSESSID -
Disallow /web/ -
Disallow /*?fr_car_normali -
Disallow */*?fr_spe -
Disallow /*?fr_sector -
Disallow /*?fr_ -
Disallow /*limit%3D* -
Disallow /*order%3D* -
Disallow /*?be_ -
Disallow /*?lu_ -
Disallow /*?es_ -
Disallow /*?nl_ -
Disallow /*?it_ -
Disallow /*?spe_ -
Disallow /*?price= -
Disallow */*?*camplink -
Disallow */*?*campname -
Disallow */aitdownloadablefiles/ -
Disallow */*car_ -
Disallow */*news%3D -
Disallow */*object%3D -
Disallow */customer/ -
Disallow /j2tajaxcheckout/ -
Disallow /*?all -
Disallow /*zipcode%3D -
Disallow /*city%3D -
Disallow /*radius%3D -
Disallow /storepickup/ -
Disallow /phone/ -
Disallow /*cmp%3D -
Disallow /productupdates/ -
Disallow /stores/ -
Disallow /quotation/ -
Disallow /search/result/ -
Disallow /retif_elasticsuite/* -
Disallow /media/product_attachment/* -
Disallow /*?fr_* -
Disallow /*?filters* -
Disallow /*?shipped_by* -
Disallow /*?filter* -
Disallow /special-pharmacie -

Other Records

Field Value
crawl-delay 10

yandex

Rule Path
Disallow /

baiduspider
baiduspider-video
baiduspider-image

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.retif.eu/media/sitemap/fr_fr/secure_sitemap_index.xml
sitemap https://www.retif.be/media/sitemap/fr_be/secure_sitemap_index.xml
sitemap https://www.retif.be/media/sitemap/nl_be/secure_sitemap_index.xml
sitemap https://www.retif.es/media/sitemap/es_es/secure_sitemap_index.xml
sitemap https://www.retif.lu/media/sitemap/fr_lu/secure_sitemap_index.xml
sitemap https://www.retif.nl/media/sitemap/nl_nl/secure_sitemap_index.xml
sitemap https://www.retif.it/media/sitemap/it_it/secure_sitemap_index.xml
sitemap https://www.retif.de/media/sitemap/de_de/secure_sitemap_index.xml

Comments

  • Website Sitemap
  • Default
  • Paths (clean URLs)
  • Paths (no clean URLs)
  • End Default - From M1
  • Crawlers Setup
  • Allowable Index
  • Mind that Allow is not an official standard
  • Directories
  • Disallow: /js/
  • Disallow: /media/
  • Disallow: /media/catalog/
  • Disallow: /media/css/
  • Disallow: /media/css_secure/
  • Disallow: /media/js/
  • Disallow: /skin/
  • Paths (clean URLs)
  • Files
  • Paths (no clean URLs)
  • Disallow: /*?p=*
  • Disallow: /*&p=*
  • Disallow: /*?price=*
  • Disallow: /*&price=*
  • Disallow: /*?color=*
  • Disallow: /*&color=*
  • Disallow: /*?limit=*
  • Disallow: /*&limit=*
  • Disallow: /*?mode=*
  • Disallow: /*&mode=*
  • Disallow: /*?order=*
  • Disallow: /*&order=*
  • Disallow: /*.js$
  • Disallow: /*.css$
  • SEO Agency 07.09.2016
  • Gautier Sept 2016