Log In Sign Up

spacesdashboard.com
robots.txt

Robots Exclusion Standard data for spacesdashboard.com

Resource Scan

Scan Details

Site Domain spacesdashboard.com
Base Domain spacesdashboard.com
Scan Status Ok
Last Scan2025-11-19T10:27:36+00:00
Next Scan 2025-12-19T10:27:36+00:00

Last Scan

Scanned2025-11-19T10:27:36+00:00
URL https://spacesdashboard.com/robots.txt
Domain IPs 104.26.2.104, 104.26.3.104, 172.67.68.198, 2606:4700:20::681a:268, 2606:4700:20::681a:368, 2606:4700:20::ac43:44c6
Response IP 104.26.2.104
Found Yes
Hash 19f64fa7d23a0d086ab91a2eca487e6ed19ddefc515361eee06d59f43c88030d
SimHash 6b5dd0007b99

Groups

*

Rule Path
Disallow /search

yandexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

twitterbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

petalbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top

Comments

  • https://yandex.com/support/webmaster/robot-workings/check-yandex-robots.html?lang=en
  • https://moz.com/help/moz-procedures/crawlers/dotbot#slow-dotbot-down
  • http://webmeup-crawler.com/
  • https://ahrefs.com/robot/
  • http://mj12bot.com/
  • https://www.semrush.com/bot/
  • https://www.turnitin.com/robot/crawlerinfo.html
  • https://webmaster.petalsearch.com/site/petalbot
Back to top