Log In Sign Up

radioforen.de
robots.txt

Robots Exclusion Standard data for radioforen.de

Resource Scan

Scan Details

Site Domain radioforen.de
Base Domain radioforen.de
Scan Status Ok
Last Scan2024-10-06T16:16:38+00:00
Next Scan 2024-10-13T16:16:38+00:00

Last Scan

Scanned2024-10-06T16:16:38+00:00
URL https://radioforen.de/robots.txt
Redirect https://www.radioforen.de/robots.txt
Redirect Domain www.radioforen.de
Redirect Base radioforen.de
Domain IPs 136.243.127.197, 2a01:4f8:d0a:677a::2
Redirect IPs 136.243.127.197, 2a01:4f8:d0a:677a::2
Response IP 136.243.127.197
Found Yes
Hash b6a10dd8cb037a0f1b84c172a24518606657f043f91e9b24c47baf14a38f0cbc
SimHash d01ece70feae

Groups

*

Rule Path
Disallow /admin.php
Disallow /account/
Disallow /attachments/
Disallow /conversations/
Disallow /help/
Disallow /login/
Disallow /members/
Disallow /posts/
Disallow /search/
Allow /

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

admantx bot
ahrefsbot
barkrowler
blexbot
bytespider
censysinspect
cherrypicker
claudebot
criteobot
dalvik
dataforseobot
dataprovider
dotbot
expanse
foregenix
go-http-client
gptbot
grub
imagesiftbot
internet-measurement
java
linkdexbot
mauibot
mj12bot
mozlila
npbot
orbbot
peer39_crawler
petalbot
proximic
python-requests
scrapy
secragon offensive agent
semrushbot
sirdatabot
sogou web spider
teleport
trendictionbot
um-ic
webzip
wp_is_mobile
xenu
zeus

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.radioforen.de/sitemap.php
Back to top

Comments

  • Allow bots
  • Slow down bots
  • Block bots
Back to top

Warnings

  • 1 invalid line.
Back to top