Log In Sign Up

derdualstudent.de
robots.txt

Robots Exclusion Standard data for derdualstudent.de

Resource Scan

Scan Details

Site Domain derdualstudent.de
Base Domain derdualstudent.de
Scan Status Ok
Last Scan2025-11-26T19:43:55+00:00
Next Scan 2025-12-03T19:43:55+00:00

Last Scan

Scanned2025-11-26T19:43:55+00:00
URL https://derdualstudent.de/robots.txt
Domain IPs 116.202.77.29, 2a01:4f8:d0a:6662::2
Response IP 116.202.77.29
Found Yes
Hash 7991e97ae202d56984ab565993b89815bfadf25108b123b1c18bfcfd7470ee17
SimHash 524c4b607620

Groups

scrapy

Rule Path
Allow /

suggybot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

kraken

Rule Path
Disallow /

genieo

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

r6_feedfetcher

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

yeti

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

seobility

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

ssearch crawler

Rule Path
Disallow /

publiclibraryarchive

Rule Path
Disallow /

publiclibraryarchive.org

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

wesee

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

netlyzer fastprobe

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

adscanner

Rule Path
Disallow /
Back to top