11teamsports.bg
robots.txt

Robots Exclusion Standard data for 11teamsports.bg

Resource Scan

Scan Details

Site Domain 11teamsports.bg
Base Domain 11teamsports.bg
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-09-22T01:48:52+00:00
Next Scan 2024-12-21T01:48:52+00:00

Last Successful Scan

Scanned2024-05-03T01:47:14+00:00
URL https://11teamsports.bg/robots.txt
Domain IPs 185.115.1.170
Response IP 185.115.1.170
Found Yes
Hash f26dbf43fd1cebd25fb2f9308c3b211c1c2b9acb67ef2ff14a42fcda3b720429
SimHash 6a05d1138633

Groups

*

Rule Path
Disallow /c/*/sizes-*2B*
Disallow /c/*/asizes-*2B*
Disallow /c/*/as-*2B*
Disallow /c/*/g_price-
Disallow /c/*/colors-*%2B*
Disallow /c/*/producer-*%2B*
Disallow /c/*/type-*%2B*
Disallow /c/*/type2-*%2B*
Disallow /c/*/shoetype-*%2B*
Disallow /c/*/rada-*%2B*
Disallow /c/*/model-*%2B*
Disallow /page/event/*
Disallow /category-v2/*
Disallow /c/*?
Disallow /s/*?
Disallow /pg/*?
Disallow /brand/*?
Allow /brand/*?collections
Disallow /basket/*
Disallow /order/*
Disallow /user/*
Disallow /category/*
Disallow /product/gear-up*
Disallow /product/similar-products*

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

amazonbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

trendkite-akashic-crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

awariorssbot
timpibot
rainbot
awariosmartbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://11teamsports.bg/sitemap.xml

Comments

  • see http://www.robotstxt.org/orig.html for documentation
  • disable filters in categories and subcategories
  • Disallow: /c/*/
  • Disallow: /c/*/st-
  • Disallow: /c/*/colors-
  • disable dynamic content and filters
  • Disallow: /*?
  • Disallow: /p/*?
  • special content
  • ajax loading
  • some delay
  • User-agent: AhrefsBot
  • Crawl-Delay: 5
  • User-agent: Pinterestbot
  • Crawl-Delay: 3
  • User-agent: bingbot
  • Crawl-Delay: 1
  • User-agent: YandexBot
  • Crawl-Delay: 10
  • User-agent: SeekportBot
  • Crawl-delay: 10
  • User-agent: Googlebot
  • Crawl-Delay: 1
  • User-agent: YandexBot
  • Crawl-Delay: 10
  • we don't use and causing non-trivial traffic