skiwebshop.es
robots.txt

Robots Exclusion Standard data for skiwebshop.es

Resource Scan

Scan Details

Site Domain skiwebshop.es
Base Domain skiwebshop.es
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-10T11:27:56+00:00
Next Scan 2026-03-10T11:27:56+00:00

Last Successful Scan

Scanned2025-04-21T10:19:50+00:00
URL https://skiwebshop.es/robots.txt
Redirect https://www.skiwebshop.es/robots.txt
Redirect Domain www.skiwebshop.es
Redirect Base skiwebshop.es
Domain IPs 104.21.1.224, 172.67.152.101, 2606:4700:3030::ac43:9865, 2606:4700:3034::6815:1e0
Redirect IPs 104.21.1.224, 172.67.152.101, 2606:4700:3030::ac43:9865, 2606:4700:3034::6815:1e0
Response IP 172.67.152.101
Found Yes
Hash 8700915f3f8bd548739c694d836cbe03c0b941cf2ee92709801ed543a599fe14
SimHash 70f1bc65c5b0

Groups

*

Rule Path
Disallow *?*%20*
Disallow *?*%25*
Disallow *?*%2C*
Disallow *?*%2C*
Disallow *?*%5B*
Disallow *?limit=*
Disallow *%26limit%3D*
Disallow *?order=*
Disallow *%26order%3D*
Disallow *?dir=*
Disallow *%26dir%3D*
Disallow *?mode=*
Disallow *%26mode%3D*
Disallow *?___store=*
Disallow *%26___store%3D*
Disallow *?___from_store=*
Disallow *%26___from_store%3D*
Disallow *?SID=*
Disallow *%26SID%3D*
Disallow *?___SID=*
Disallow *%26___SID%3D*
Disallow */sort-by/
Disallow */sort-direction/
Disallow */limit/
Disallow */dir/
Disallow */order/
Disallow */catalogsearch/
Disallow */search/
Disallow */customer/
Disallow */checkout/
Disallow */errors/
Disallow */onestepcheckout/
Disallow /catalog/product/view/
Disallow /catalog/category/view/
Disallow /*?*product_list_mode=
Disallow /*?*product_list_order=
Disallow /*?*product_list_limit=
Disallow /*?*product_list_dir=

Other Records

Field Value
crawl-delay 2

oai-searchbot
chatgpt-user
perplexitybot
andibot
phindbot
youbot

Rule Path
Allow /

gptbot
ccbot
google-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.skiwebshop.es/media/sitemap_es_index.xml

Comments

  • MageHost: prevent Google from going wild on filter/sorting params
  • Allow AI search and agent use
  • Disallow AI training data collection