Log In Sign Up

outdoorxl.pl
robots.txt

Robots Exclusion Standard data for outdoorxl.pl

Resource Scan

Scan Details

Site Domain outdoorxl.pl
Base Domain outdoorxl.pl
Scan Status Ok
Last Scan3/28/2025, 4:09:20 AM
Next Scan 4/27/2025, 4:09:20 AM

Last Scan

Scanned3/28/2025, 4:09:20 AM
URL https://outdoorxl.pl/robots.txt
Redirect https://www.outdoorxl.pl/robots.txt
Redirect Domain www.outdoorxl.pl
Redirect Base outdoorxl.pl
Domain IPs 104.21.82.157, 172.67.158.243, 2606:4700:3031::6815:529d, 2606:4700:3035::ac43:9ef3
Redirect IPs 104.21.82.157, 172.67.158.243, 2606:4700:3031::6815:529d, 2606:4700:3035::ac43:9ef3
Response IP 104.21.82.157
Found Yes
Hash 618354eda35bfde782044b380770b3263a9d132dc492988b8920e7e8d8938f9e
SimHash a106f84547f8

Groups

*

Rule Path
Allow /*?p=
Disallow /404/
Disallow /app/
Disallow /cgi-bin/
Disallow /downloader/
Disallow /errors/
Disallow /includes/
Disallow /magento/
Disallow /media/captcha/
Disallow /media/customer/
Disallow /media/dhl/
Disallow /media/downloadable/
Disallow /media/import/
Disallow /media/pdf/
Disallow /media/sales/
Disallow /media/tmp/
Disallow /media/xmlconnect/
Disallow /pkginfo/
Disallow /report/
Disallow /scripts/
Disallow /shell/
Disallow /stats/
Disallow /var/
Disallow /kb/
Disallow /index.php/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalog/product/gallery/
Disallow /catalogsearch/
Disallow /control/
Disallow /contacts/
Disallow /customer/
Disallow /customize/
Disallow /newsletter/
Disallow /poll/
Disallow /review/
Disallow /sendfriend/
Disallow /tag/
Disallow /wishlist/
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /STATUS.txt
Disallow /turpentine/
Disallow /*?p=*&
Disallow /*.php$
Disallow /*?SID=
Disallow /*.cfm
Disallow /?___from_store=
Disallow /*___from_store%3D
Disallow /*?dir=
Disallow /*?mode=
Disallow /*?limit=
Disallow /*?___store=
Disallow /*___store%3D
Disallow /configurator/binding/selector/
Disallow /amconf/ajax/
Disallow /p010_performance/stock/
Disallow *?*%20*
Disallow *?*%25*
Disallow *?*%2C*
Disallow *?*%2C*
Disallow *?*%5B*
Disallow *?limit=*
Disallow *%26limit%3D*
Disallow *?order=*
Disallow *%26order%3D*
Disallow *?dir=*
Disallow *%26dir%3D*
Disallow *?mode=*
Disallow *%26mode%3D*
Disallow *?___store=*
Disallow *%26___store%3D*
Disallow *?___from_store=*
Disallow *%26___from_store%3D*
Disallow *?SID=*
Disallow *%26SID%3D*
Disallow *?___SID=*
Disallow *%26___SID%3D*
Disallow */sort-by/
Disallow */sort-direction/
Disallow */limit/
Disallow */dir/
Disallow */order/
Disallow */catalogsearch/
Disallow */search/
Disallow */customer/
Disallow */checkout/
Disallow */errors/
Disallow */onestepcheckout/
Disallow *?manufacturer=*
Disallow *?clothingsize=*
Disallow *?price=*
Disallow *?ranker_type_ski=*
Disallow *?ski_length=*
Disallow *?ranker_snowboard_flex=*
Disallow *?ranker_skischoen_flex=*
Disallow *?ranker_schoen_volume=*
Disallow *?ranker_ski_breedte_ob=*
Disallow *?ranker_gewicht_per_ski=*
Disallow *?ranker_capaciteit=*
Disallow *?ranker_materiaal_buitentent=*
Disallow *?color_group=*
Disallow *?snowboard_width=*
Disallow *?shoesize=*
Disallow *?price=*
Disallow *%26manufacturer%3D*
Disallow *%26clothingsize%3D*
Disallow *%26price%3D*
Disallow *%26ranker_type_ski%3D*
Disallow *%26ski_length%3D*
Disallow *%26ranker_snowboard_flex%3D*
Disallow *%26ranker_skischoen_flex%3D*
Disallow *%26ranker_schoen_volume%3D*
Disallow *%26ranker_ski_breedte_ob%3D*
Disallow *%26ranker_gewicht_per_ski%3D*
Disallow *%26ranker_capaciteit%3D*
Disallow *%26ranker_materiaal_buitentent%3D*
Disallow *%26color_group%3D*
Disallow *%26snowboard_width%3D*
Disallow *%26shoesize%3D*
Disallow *%26price%3D*

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top

Other Records

Field Value
sitemap https://www.outdoorxl.pl/sitemap.xml
Back to top

Comments

  • robots.txt for Magento / v1.3 2015-04-23 / Peeter Marvet
  • based on http://inchoo.net/ecommerce/ultimate-magento-robots-txt-file-examples/ and http://www.byte.nl/blog/magento-robots-txt/
  • comment and clone at https://gist.github.com/petskratt/016c9dbf159a81b9d6aa
  • Keep in mind that by standard robots.txt should NOT contain empty lines, except between UA blocks!
  • Sitemap (uncomment, change and add language/shop specific sitemaps, if running on multiple domains
  • keep in mind sitemap can only point to own domain so something like sitemapindex.php is needed)
  • Google Image Crawler Setup - having crawler-specific sections makes it ignore generic e.g *
  • User-agent: Googlebot-Image
  • Crawlers Setup
  • Allow paging (unless has more params, disallowed below)
  • Directories
  • Disallow: /media/
  • Disallow: /media/catalog/
  • Disallow: /media/wysiwyg/
  • Disallow: /skin/
  • Paths (if using shop id in URL must prefix with * or copy for each)
  • Files
  • prevent google from crawling all these turpentine
  • Paths (no clean URLs)
  • Binding selector
  • MageHost: prevent Google from going wild on filter/sorting params
  • OutdoorXL Disable Filtered Pages
Back to top