Log In Sign Up

bo.de
robots.txt

Robots Exclusion Standard data for bo.de

Resource Scan

Scan Details

Site Domain bo.de
Base Domain bo.de
Scan Status Ok
Last Scan2025-12-08T13:14:56+00:00
Next Scan 2025-12-09T13:14:56+00:00

Last Scan

Scanned2025-12-08T13:14:56+00:00
URL https://bo.de/robots.txt
Domain IPs 151.101.131.7, 151.101.195.7, 151.101.3.7, 151.101.67.7, 2a04:4e42:200::775, 2a04:4e42:400::775, 2a04:4e42:600::775, 2a04:4e42::775
Response IP 151.101.3.7
Found Yes
Hash 5e149940a12fab0a62928697a46604cd7ee101d1cb9390d74fd846ae5b4be3fb
SimHash 709c5195e417

Groups

*

Rule Path
Disallow /ghost/
Disallow /email/
Disallow /members/api/comments/counts/
Disallow /r/
Disallow /webmentions/receive/

applebot-extended
bytespider
ccbot
claudebot
cohere-training-data-crawler
diffbot
facebookbot
google-extended
gptbot
meta-externalagent
omgili
timpibot

Rule Path
Disallow /

amazonbot
oai-searchbot
perplexitybot
youbot

Rule Path
Disallow /

chatgpt-user
anthropic-ai
claude-web
cohere-ai

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://bo.de/sitemap.xml
Back to top

Comments

  • DarkVisitors Category AI DataScrapers
  • DarkVisitors Category AI Search Crawler
  • DarkVisitors Category AI Assistants
  • DarkVisitors Category Undocumented AI Agents
Back to top