Log In Sign Up

11teamsports.cz
robots.txt

Robots Exclusion Standard data for 11teamsports.cz

Resource Scan

Scan Details

Site Domain 11teamsports.cz
Base Domain 11teamsports.cz
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-05-29T22:57:21+00:00
Next Scan 2024-06-28T22:57:21+00:00

Last Successful Scan

Scanned2024-04-07T22:56:28+00:00
URL https://11teamsports.cz/robots.txt
Domain IPs 185.115.1.170
Response IP 185.115.1.170
Found Yes
Hash 0c0f8f8ed28f7c75d68bfa2b41851ce6575fa9f26c15b19e8288d5ed2af74182
SimHash 6a0cd1108631

Groups

*

Rule Path
Disallow /c/*/sizes-*2B*
Disallow /c/*/asizes-*2B*
Disallow /c/*/as-*2B*
Disallow /c/*/g_price-
Disallow /c/*/colors-*%2B*
Disallow /c/*/producer-*%2B*
Disallow /c/*/type-*%2B*
Disallow /c/*/type2-*%2B*
Disallow /c/*/shoetype-*%2B*
Disallow /c/*/rada-*%2B*
Disallow /c/*/model-*%2B*
Disallow /page/event/*
Disallow /category-v2/*
Disallow /c/*?
Disallow /s/*?
Disallow /pg/*?
Disallow /brand/*?
Allow /brand/*?collections
Disallow /basket/*
Disallow /order/*
Disallow /user/*
Disallow /category/*
Disallow /product/gear-up*
Disallow /product/similar-products*

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

amazonbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

trendkite-akashic-crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

awariorssbot
timpibot
rainbot
awariosmartbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://11teamsports.cz/sitemap.xml
Back to top

Comments

  • see http://www.robotstxt.org/orig.html for documentation
  • disable filters in categories and subcategories
  • Disallow: /c/*/
  • Disallow: /c/*/st-
  • Disallow: /c/*/colors-
  • disable dynamic content and filters
  • Disallow: /*?
  • Disallow: /p/*?
  • special content
  • ajax loading
  • some delay
  • User-agent: AhrefsBot
  • Crawl-Delay: 5
  • User-agent: Pinterestbot
  • Crawl-Delay: 3
  • User-agent: bingbot
  • Crawl-Delay: 1
  • User-agent: YandexBot
  • Crawl-Delay: 10
  • User-agent: SeekportBot
  • Crawl-delay: 10
  • User-agent: Googlebot
  • Crawl-Delay: 1
  • User-agent: YandexBot
  • Crawl-Delay: 10
  • we don't use and causing non-trivial traffic
Back to top