Log In Sign Up

bsky.app
robots.txt

Robots Exclusion Standard data for bsky.app

Resource Scan

Scan Details

Site Domain bsky.app
Base Domain bsky.app
Scan Status Ok
Last Scan2024-12-02T16:38:39+00:00
Next Scan 2024-12-16T16:38:39+00:00

Last Scan

Scanned2024-12-02T16:38:39+00:00
URL https://bsky.app/robots.txt
Domain IPs 18.220.180.251, 18.220.24.239, 3.130.37.203
Response IP 18.220.180.251
Found Yes
Hash 1aadb55636f887e2bc10ea9b73f86df10c5081052693f97656cd6686c2460288
SimHash ae128e504547

Groups

*

Rule Path
Allow /
Back to top

Comments

  • Hello Friends!
  • If you are considering bulk or automated crawling, you may want to look in
  • to our protocol (API), including a firehose of updates. See: https://atproto.com/
  • By default, may crawl anything on this domain. HTTP 429 ("backoff") status
  • codes are used for rate-limiting. Up to a handful concurrent requests should
  • be ok.
Back to top