Log In Sign Up

administrator.community
robots.txt

Robots Exclusion Standard data for administrator.community

Resource Scan

Scan Details

Site Domain administrator.community
Base Domain administrator.community
Scan Status Ok
Last Scan2025-09-13T14:15:12+00:00
Next Scan 2025-09-20T14:15:12+00:00

Last Scan

Scanned2025-09-13T14:15:12+00:00
URL http://administrator.community/robots.txt
Redirect https://administrator.de/robots.txt
Redirect Domain administrator.de
Redirect Base administrator.de
Domain IPs 89.31.143.90
Redirect IPs 82.149.225.19
Response IP 82.149.225.19
Found Yes
Hash 56051cd4abd4ab7eb459f3b8cc424953537f2c2f920fc767077463296404eff3
SimHash 7818d119cfe8

Groups

gptbot
chatgpt-user
chatgpt-bot
openaibot

Rule Path
Disallow /

anthropic-ai
claude-web
claude-bot

Rule Path
Disallow /

google-extended
bard-bot
gemini-bot

Rule Path
Disallow /

bingpreview
msnbot-media
copilotbot

Rule Path
Disallow /

facebookbot
facebookexternalhit
meta-externalagent

Rule Path
Disallow /

ccbot
perplexitybot
amazonbot
applebot-extended
applebot
cohere-ai
youbot
diffbot

Rule Path
Disallow /

twitterbot
linkedinbot
whatsapp
telegrambot

Rule Path
Disallow /

bytespider
baiduspider
sogou
yisouspider
bytedance
petalbot

Rule Path
Disallow /

dataforseobot
mj12bot
scrapy
magpie-crawler
omgilibot

Rule Path
Disallow /

commoncrawl
webcrawler
imagesiftbot

Rule Path
Disallow /

googlebot
bingbot
duckduckbot
yandexbot
slurp

Rule Path
Disallow /settings/
Disallow /api/
Disallow /members/dashboard/
Disallow /members/notifications/
Disallow /moderators/reports/
Disallow /connections/
Disallow /r/
Disallow /forum/?search=
Disallow /comments/?search=
Disallow /members/?search=
Disallow /*?sort=
Disallow /*?filter=
Disallow /*?time=
Disallow /*?lang=
Disallow /*?action=
Disallow /*?distance=
Disallow /*?uri=
Disallow /*?topic=
Disallow /*?target=
Disallow /*?target_id=
Allow /

*

Rule Path
Disallow /settings/
Disallow /api/
Disallow /members/dashboard/
Disallow /members/notifications/
Disallow /moderators/reports/
Disallow /connections/
Disallow /r/
Disallow /forum/?search=
Disallow /comments/?search=
Disallow /members/?search=
Disallow /*?sort=
Disallow /*?filter=
Disallow /*?time=
Disallow /*?lang=
Disallow /*?action=
Disallow /*?distance=
Disallow /*?uri=
Disallow /*?topic=
Disallow /*?target=
Disallow /*?target_id=
Allow /
Back to top

Other Records

Field Value
sitemap https://administrator.de/sitemap/sitemap_index.xml
Back to top

Comments

  • robots.txt for administrator.de
  • Last Updated: 20250814
  • ================================================
  • LICENSING NOTICE - IMPORTANT
  • ================================================
  • The use of content from administrator.de for
  • AI training or commercial AI applications is
  • STRICTLY PROHIBITED without written license agreement.
  • License inquiries: licence@administrator.de
  • Terms of Service: https://administrator.de/help/80 (EN: https://administrator.de/help/80?lang=en)
  • Privacy Policy: https://administrator.de/help/79 (EN: https://administrator.de/help/79?lang=en)
  • AI Licensing Info: https://administrator.de/help/82 (EN: https://administrator.de/help/82?lang=en)
  • ================================================
  • ================================================
  • AI CRAWLERS - COMPLETELY BLOCKED without license
  • ================================================
  • OpenAI Crawlers
  • Anthropic (Claude) Crawlers
  • Google AI Crawlers
  • Microsoft/Bing AI Crawlers
  • Meta/Facebook AI Crawlers
  • Other Major AI Companies
  • Social Media AI Crawlers
  • Chinese AI Crawlers
  • SEO/Data Mining Crawlers
  • User-agent: SemrushBot
  • User-agent: AhrefsBot
  • Archive/Research Crawlers (often used for AI training)
  • Generic AI Pattern Blocking
  • ================================================
  • LEGITIMATE SEARCH ENGINES - ALLOWED
  • ================================================
  • ================================================
  • DEFAULT RULES
  • ================================================
  • Sitemap
  • ================================================
  • LEGAL NOTICE - EU DSM DIRECTIVE
  • ================================================
  • According to Article 4 of EU Directive 2019/790 (DSM Directive)
  • we explicitly opt-out from text and data mining (TDM)
  • for commercial purposes without license agreement.
  • TDM-Reservation: 1
  • TDM-Policy: https://administrator.de/help/82 (EN: https://administrator.de/help/82?lang=en)
  • Terms of Service: https://administrator.de/help/80 (EN: https://administrator.de/help/80?lang=en)
  • Privacy Policy: https://administrator.de/help/79 (EN: https://administrator.de/help/79?lang=en)
Back to top

Warnings

  • 10 invalid lines.
Back to top