Log In Sign Up

rlsbb.cc
robots.txt

Robots Exclusion Standard data for rlsbb.cc

Resource Scan

Scan Details

Site Domain rlsbb.cc
Base Domain rlsbb.cc
Scan Status Ok
Last Scan2025-12-06T21:10:34+00:00
Next Scan 2026-01-05T21:10:34+00:00

Last Scan

Scanned2025-12-06T21:10:34+00:00
URL https://rlsbb.cc/robots.txt
Domain IPs 104.21.83.46, 172.67.214.8, 2606:4700:3030::6815:532e, 2606:4700:3030::ac43:d608
Response IP 104.21.83.46
Found Yes
Hash 046c1ba747389e7fa5d91ecc6f516555e2df2e93035c4f1e8cec5abe3b35bf24
SimHash 120edc1145f4

Groups

*

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10
Back to top

Comments

  • Specific disallow for known AI data collectors (if any are identified)
  • You can also explicitly allow good crawlers like search engines
  • Crawl-delay directive to slow down aggressive crawlers (not all support this)
Back to top