bottleneckcalculator.to
robots.txt

Robots Exclusion Standard data for bottleneckcalculator.to

Resource Scan

Scan Details

Site Domain bottleneckcalculator.to
Base Domain bottleneckcalculator.to
Scan Status Ok
Last Scan2025-09-11T11:13:48+00:00
Next Scan 2025-09-18T11:13:48+00:00

Last Scan

Scanned2025-09-11T11:13:48+00:00
URL https://bottleneckcalculator.to/robots.txt
Domain IPs 209.74.74.27
Response IP 209.74.74.27
Found Yes
Hash 6629a0e0f46f874f07c028dd0f58553c27443bd06bef39cc4da7e2a69f08aa85
SimHash 0b5ede90d6f0

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-news

Rule Path
Disallow

googlebot-video

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

adsbot-google-mobile

Rule Path
Disallow

*

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

yandex

Rule Path
Disallow

baiduspider

Rule Path
Disallow /

spbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

seekport

Rule Path
Disallow /

sogou

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow

Other Records

Field Value
sitemap https://bottleneckcalculator.to/map.xml

Comments

  • robots.txt for example.com
  • Last updated: 2025-07-19
  • Allow all Google bots
  • Slow down generic/basic crawlers
  • Block known bad crawlers
  • Sitemap