scottusa.com
robots.txt

Robots Exclusion Standard data for scottusa.com

Resource Scan

Scan Details

Site Domain scottusa.com
Base Domain scottusa.com
Scan Status Ok
Last Scan2024-10-28T08:02:21+00:00
Next Scan 2024-11-27T08:02:21+00:00

Last Scan

Scanned2024-10-28T08:02:21+00:00
URL https://scottusa.com/robots.txt
Redirect https://www.scott-sports.com/robots.txt
Redirect Domain www.scott-sports.com
Redirect Base scott-sports.com
Domain IPs 108.157.254.34, 108.157.254.41, 108.157.254.77, 108.157.254.92
Redirect IPs 107.154.77.125
Response IP 107.154.77.125
Found Yes
Hash ba206f4bcf5477613bc9edb0d2609c80f671c018729c2a3278c49389cf67744b
SimHash 2cd0b1f6f97a

Groups

*

Rule Path
Disallow /*/country*
Allow /*/country$
Disallow /*/cart
Disallow /*/checkout
Disallow /*/my-account
Disallow /*/releaseVersion.txt
Disallow /*/buildDescription.txt
Disallow /*/dealers/locator
Disallow /*?q=
Disallow /*%26q%3D
Allow /*?q=%3Arelevant
Allow /*%26q%3D%3Arelevant
Disallow /*?number=
Disallow /*%26number%3D
Allow /*?number=24
Allow /*%26number%3D24
Allow /*?page=
Allow /*%26page%3D
Allow /global/en/
Allow /at/de/
Allow /be/fr/
Allow /be/en/
Allow /ch/de/
Allow /ch/fr/
Allow /ch/it/
Allow /cz/cs/
Allow /de/de/
Allow /dk/en/
Allow /es/es/
Allow /fr/fr/
Allow /gb/en/
Allow /it/it/
Allow /nl/en/
Allow /pl/pl/
Allow /pt/pt/
Allow /se/en/
Allow /ar/es/
Allow /br/pt/
Allow /ca/en/
Allow /ca/fr/
Allow /cl/es/
Allow /co/es/
Allow /mx/es/
Allow /us/en/
Allow /au/en/
Allow /nz/en/
Allow /cn/zh/
Allow /in/en/
Allow /jp/ja/
Allow /kr/ko/
Allow /za/en/
Disallow /ar/
Disallow /at/
Disallow /au/
Disallow /be/
Disallow /bg/
Disallow /br/
Disallow /ca/
Disallow /ch/
Disallow /cl/
Disallow /cn/
Disallow /co/
Disallow /cr/
Disallow /cy/
Disallow /cz/
Disallow /de/
Disallow /dk/
Disallow /do/
Disallow /ec/
Disallow /ee/
Disallow /es/
Disallow /fi/
Disallow /fr/
Disallow /gb/
Disallow /global/
Disallow /gt/
Disallow /hk/
Disallow /hr/
Disallow /ie/
Disallow /il/
Disallow /in/
Disallow /is/
Disallow /it/
Disallow /jp/
Disallow /kr/
Disallow /lu/
Disallow /lv/
Disallow /mx/
Disallow /nl/
Disallow /no/
Disallow /nz/
Disallow /pa/
Disallow /pe/
Disallow /pl/
Disallow /pt/
Disallow /re/
Disallow /rs/
Disallow /ru/
Disallow /se/
Disallow /sg/
Disallow /si/
Disallow /sk/
Disallow /tw/
Disallow /ua/
Disallow /us/
Disallow /uy/
Disallow /za/

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

semrushbot

Rule Path
Allow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.scott-sports.com/at/de/sitemap.xml
sitemap https://www.scott-sports.com/be/fr/sitemap.xml
sitemap https://www.scott-sports.com/be/en/sitemap.xml
sitemap https://www.scott-sports.com/ch/de/sitemap.xml
sitemap https://www.scott-sports.com/ch/fr/sitemap.xml
sitemap https://www.scott-sports.com/ch/it/sitemap.xml
sitemap https://www.scott-sports.com/cz/cs/sitemap.xml
sitemap https://www.scott-sports.com/de/de/sitemap.xml
sitemap https://www.scott-sports.com/dk/en/sitemap.xml
sitemap https://www.scott-sports.com/es/es/sitemap.xml
sitemap https://www.scott-sports.com/fr/fr/sitemap.xml
sitemap https://www.scott-sports.com/gb/en/sitemap.xml
sitemap https://www.scott-sports.com/it/it/sitemap.xml
sitemap https://www.scott-sports.com/nl/en/sitemap.xml
sitemap https://www.scott-sports.com/pl/pl/sitemap.xml
sitemap https://www.scott-sports.com/pt/pt/sitemap.xml
sitemap https://www.scott-sports.com/se/en/sitemap.xml
sitemap https://www.scott-sports.com/ar/es/sitemap.xml
sitemap https://www.scott-sports.com/br/pt/sitemap.xml
sitemap https://www.scott-sports.com/ca/en/sitemap.xml
sitemap https://www.scott-sports.com/ca/fr/sitemap.xml
sitemap https://www.scott-sports.com/cl/es/sitemap.xml
sitemap https://www.scott-sports.com/co/es/sitemap.xml
sitemap https://www.scott-sports.com/mx/es/sitemap.xml
sitemap https://www.scott-sports.com/us/en/sitemap.xml
sitemap https://www.scott-sports.com/au/en/sitemap.xml
sitemap https://www.scott-sports.com/nz/en/sitemap.xml
sitemap https://www.scott-sports.com/cn/zh/sitemap.xml
sitemap https://www.scott-sports.com/in/en/sitemap.xml
sitemap https://www.scott-sports.com/jp/ja/sitemap.xml
sitemap https://www.scott-sports.com/kr/ko/sitemap.xml
sitemap https://www.scott-sports.com/za/en/sitemap.xml
sitemap https://www.scott-sports.com/global/en/sitemap.xml

Comments

  • For all robots
  • Block access to specific groups of pages
  • PLP filters
  • Avoid bad mix country/lang
  • Europe
  • Americas
  • Oceania
  • Asia
  • Africa
  • Disallow
  • Allow search crawlers to discover the sitemap
  • europe
  • americas
  • oceania
  • asia
  • africa
  • global
  • Block CazoodleBot as it does not present correct accept content headers
  • Block MJ12bot as it is just noise
  • Block dotbot as it cannot parse base urls properly
  • Block Gigabot
  • Allow Marketing tool SEMRush bot
  • Block AhrefsBot
  • Block Blexbot