restposten24.de
robots.txt

Robots Exclusion Standard data for restposten24.de

Resource Scan

Scan Details

Site Domain restposten24.de
Base Domain restposten24.de
Scan Status Ok
Last Scan2026-01-17T07:46:45+00:00
Next Scan 2026-01-24T07:46:45+00:00

Last Scan

Scanned2026-01-17T07:46:45+00:00
URL https://restposten24.de/robots.txt
Redirect https://www.restposten24.de/robots.txt
Redirect Domain www.restposten24.de
Redirect Base restposten24.de
Domain IPs 85.214.202.90
Redirect IPs 85.214.202.90
Response IP 85.214.202.90
Found Yes
Hash 7a2eff8d70fe247789f716f519a2bccc7d2ef28472c04496e5f4007199eeb078
SimHash 681dd551d311

Groups

*

Rule Path
Disallow /compiled/
Disallow /print/
Disallow /data/
Disallow /modules/rp24_global/ajax/

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 12

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

tigerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

oppo a33

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

nekstbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow

googlebot

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

msnbot

Rule Path
Disallow

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

slurp

Rule Path
Disallow

yahoo! slurp

Rule Path
Disallow

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 25