speedyservices.com
robots.txt

Robots Exclusion Standard data for speedyservices.com

Resource Scan

Scan Details

Site Domain speedyservices.com
Base Domain speedyservices.com
Scan Status Ok
Last Scan2024-10-01T05:10:35+00:00
Next Scan 2024-10-15T05:10:35+00:00

Last Scan

Scanned2024-10-01T05:10:35+00:00
URL https://speedyservices.com/robots.txt
Redirect https://www.speedyservices.com/robots.txt
Redirect Domain www.speedyservices.com
Redirect Base speedyservices.com
Domain IPs 185.58.89.177
Redirect IPs 185.58.89.177
Response IP 185.58.89.177
Found Yes
Hash 2ba8da5c2df0d1f8bb816031b53b42deb54b9c84c2feddb9178f3750b98c2ca8
SimHash 5e75d7368f7a

Groups

*

Rule Path
Disallow /legacy
Disallow /legacy/mw
Disallow /login
Disallow /guest-login
Disallow /shortlist
Disallow /bcms-speedycatalogue/Checkout/GetSplitBasketModel
Disallow /bcms-speedycatalogue/Catalogue/AddToShortlist
Disallow /search
Disallow /set-password
Disallow /search/getsearch
Disallow /search/getsearchsuggestions
Disallow /payments/m
Disallow /bcms-speedycatalogue/Checkout/DeleteProduct
Disallow /checkout
Disallow /autodiscover/autodiscover.svc/WSSecurity
Disallow /users/set-password
Disallow /vat-state/switch/
Disallow /vat-state/switch/Include
Disallow /bcms-speedycore/BusinessTypeSwitch/Index

Other Records

Field Value
crawl-delay 5

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

petalbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

blp_bbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seznambot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.speedyservices.com/sitemap.xml

Comments

  • Bot for Huawei search engine
  • Associated with Bloomberg News, likely involved in web scraping or data gathering for financial purposes
  • SeznamBot working for Czech internet company
  • Speedyservices is not working in China and Russia
  • Used for SEO purposes by external organizations, can consume a lot of bandwidth without providing any direct benefit
  • Sending too many requests, while we do not have any partnership with Amazon
  • AI bot