reservistenverband.de
robots.txt

Robots Exclusion Standard data for reservistenverband.de

Resource Scan

Scan Details

Site Domain reservistenverband.de
Base Domain reservistenverband.de
Scan Status Ok
Last Scan2024-11-05T01:58:20+00:00
Next Scan 2024-12-05T01:58:20+00:00

Last Scan

Scanned2024-11-05T01:58:20+00:00
URL https://reservistenverband.de/robots.txt
Redirect https://www.reservistenverband.de/robots.txt
Redirect Domain www.reservistenverband.de
Redirect Base reservistenverband.de
Domain IPs 62.112.72.96
Redirect IPs 62.112.72.96
Response IP 62.112.72.96
Found Yes
Hash 84e21f42884b22834c40116eb1e0d7c72602c57f2919491101de2700310db103
SimHash 1218dc924820

Groups

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

xovi

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

deepcrawl

Rule Path
Disallow /

nutch

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

semrushbot-seoab

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 5000