bikestory.com.ua
robots.txt

Robots Exclusion Standard data for bikestory.com.ua

Resource Scan

Scan Details

Site Domain bikestory.com.ua
Base Domain bikestory.com.ua
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-02-24T17:11:55+00:00
Next Scan 2025-04-25T17:11:55+00:00

Last Successful Scan

Scanned2024-12-04T15:04:24+00:00
URL https://bikestory.com.ua/robots.txt
Domain IPs 104.21.18.70, 172.67.180.199, 2606:4700:3033::6815:1246, 2606:4700:3033::ac43:b4c7
Response IP 172.67.180.199
Found Yes
Hash 38d8b10145543bfe4654b3ff47d9bf2ba80647f0690cc923a22cc760072f3cde
SimHash 2231fa43c310

Groups

*

Rule Path
Allow */*.js
Allow */*.css
Allow */*.png
Allow */*.jpg
Allow */*.jpeg
Disallow /?s=*
Disallow *?s=
Disallow *%26s%3D
Disallow /?orderby=*
Disallow /my-account/
Disallow /uncategorized/
Disallow /checkout/
Disallow /cart/
Disallow /?filter*
Disallow /terms-and-conditions/
Disallow /shipping/
Disallow /refund-policy/
Disallow /privacy-policy/
Disallow /my-account/
Disallow /wp-admin/
Disallow /cgi-bin
Disallow /search/
Disallow /author/
Disallow /users/
Disallow *page*
Disallow *orderby
Disallow *query_type
Disallow *product_count
Disallow */trackback
Disallow /my-account
Disallow *feed
Disallow */feed
Disallow *rss
Disallow */embed
Disallow */wlwmanifest.xml
Disallow /xmlrpc.php
Disallow *utm*%3D
Disallow *openstat%3D
Disallow /uncategorized/
Disallow *currency_id
Disallow *sv-presence*
Disallow /en/*
Disallow *fbclid*
Disallow *fbclid
Disallow *srsltid*
Disallow *gclid*

amazonbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 120

semrushbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 120

ahrefsbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 120

webalta

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 120

teoma

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 4

megaindex

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 120

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 6

yahoo

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 4

Other Records

Field Value
sitemap https://bikestory.com.ua/sitemap_index.xml
sitemap https://bikestory.com.ua/ua/sitemap_index.xml

Warnings

  • `host` is not a known field.