Log In Sign Up

drexcode.com
robots.txt

Robots Exclusion Standard data for drexcode.com

Resource Scan

Scan Details

Site Domain drexcode.com
Base Domain drexcode.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-14T21:04:04+00:00
Next Scan 2026-02-12T21:04:04+00:00

Last Successful Scan

Scanned2025-06-23T23:41:26+00:00
URL https://drexcode.com/robots.txt
Domain IPs 104.21.52.201
Response IP 104.21.52.201
Found Yes
Hash 22e192f5d36528966af8540c48624c38553026c90d3f717079e1470c02a8cd54
SimHash 67103378c7be

Groups

*

Rule Path
Disallow /wp/wp-admin/
Disallow /wp/wp-includes/
Disallow /wp/wp-content/
Disallow /wp-content/
Allow /wp/wp-content/uploads/
Disallow *?r=*
Disallow */filter/*
Disallow */wishlist/*
Disallow *%26filter*
Disallow *?filter*
Disallow *?query*
Disallow *%26query*
Disallow *landing*
Disallow *review/product*
Disallow *ebizautoresponder*
Disallow *addPersonToList*
Disallow *?q=*
Disallow *?le_lunghezza=*
Disallow *?custom_price=*
Disallow *?dx_size=*
Disallow *?dx_occasioni=*
Disallow *?dx_designer=*
Disallow *?color=*
Disallow *?le_composizione=*
Disallow *?maniche=*
Disallow *?le_scollatura=*
Disallow /catalogsearch/
Disallow /*?*
Disallow *?custom_price_*=*
Disallow /skin/frontend/drexcode/new/fonts/DidotLTStd-Roman/*.html
Allow */fr/*
Disallow /*?*
Back to top

Other Records

Field Value
sitemap https://drexcode.com/sitemap/sitemap.xml
Back to top

Comments

  • Robots.txt for Magento Community and Enterprise
  • GENERAL SETTINGS
  • Enables robots.txt rules for all crawlers
  • Crawl-delay parameter: the number of seconds you want to wait between successful requests to the same server.
  • Set a crawl rate, if your server's traffic problems. Please note that Google ignore crawl-delay setting in Robots.txt. You can set up this in Google Webmaster tool
  • Crawl-delay: 30
  • Magento sitemap: URL to your sitemap file in Magento
  • Settings that relate to the UNDER CONSTRUCTION
  • Do not allow indexing files and folders that are required during development: CVS, SVN directory and dump files
  • GENERAL SETTINGS For MAGENTO
  • Do not index the page Magento admin
  • Do not index the general technical Magento directory
  • Do not index the shared files Magento
  • MAGENTO SEA IMPROVEMENT
  • Do not index the page subcategories that are sorted or filtered.
  • Do not index the second copy of the home page (example.com / index.php /). Un-comment only if you have activated Magento SEO URLs.
  • Disallow: / index.php /
  • Do not index the link from the session ID
  • Do not index the page checkout and user account
  • added 12/08/19
  • Do not index the search page and CEO, non-optimized link categories
  • Server Settings
  • Do not index the general technical directories and files on a server
  • SETTINGS Image indexing
  • Optional: If you do not want to Google and Bing to index your images
  • User-agent: Googlebot-Image
  • Disallow: /
  • User-agent: msnbot-media
  • Disallow: /
  • Wordpress
  • Do not index specific parameters
  • added 12/08/19
  • added 02/10/19
  • do no index html page for font
  • allow google to crawl all FR url
Back to top

Warnings

  • 46 invalid lines.
Back to top