Log In Sign Up

selbst.de
robots.txt

Robots Exclusion Standard data for selbst.de

Resource Scan

Scan Details

Site Domain selbst.de
Base Domain selbst.de
Scan Status Ok
Last Scan2025-11-30T02:47:01+00:00
Next Scan 2025-12-07T02:47:01+00:00

Last Scan

Scanned2025-11-30T02:47:01+00:00
URL https://selbst.de/robots.txt
Redirect https://www.selbst.de/robots.txt
Redirect Domain www.selbst.de
Redirect Base selbst.de
Domain IPs 18.159.14.252, 35.156.58.21, 35.157.92.100
Redirect IPs 18.159.14.252, 35.156.58.21, 35.157.92.100
Response IP 35.156.58.21
Found Yes
Hash 42710858724b3cb0e8a67468fc46c3426cc10e789d1571780c62a231d0994aa6
SimHash f2142d11dca5

Groups

*

Rule Path
Allow /
Disallow /fonts/whitelabel/
Disallow /images/whitelabel/
Disallow /status
Disallow /suche
Disallow *?exclude*=*
Disallow /*%26exclude*%3D*
Disallow *?print=*
Disallow *%26print%3D*
Disallow /wer-streamt?query=

gptbot
chatgpt-user
oai-searchbot
perplexitybot
perplexity-user
claude-web
claudebot
claude-user
google-extended
bytespider
timpibot
petalbot
neevaai
blexbot
ccbot
yandexbot
baiduspider
magpie-crawler
dataforseobot
diffbot
meta-externalagent
applebot-extended
cloudvertexbot
deepseekbot
cohere-training-data-crawler
pangubot
ai2bot
omgili
webzio-extended
scrapy
llc
imagesiftbot
mj12bot
yeti
deepseek agent
googleagent-mariner
chatgpt-operator
python-urllib
fuzz faster u fool v2.1.0-dev
anthropic-ai

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.selbst.de/sitemap.xml
Back to top

Comments

  • Legal notice: We expressly reserves the right to use its content for commercial text and data mining
  • The use of robots or other automated means to access our site or collect or mine data without the express permission of us is strictly prohibited.
  • We may, in its discretion, permit certain automated access to certain pages,
  • If you would like to apply for permission to crawl us, collect or use data, please email info@bauerxcel.de
Back to top