nivito.de
robots.txt

Robots Exclusion Standard data for nivito.de

Resource Scan

Scan Details

Site Domain nivito.de
Base Domain nivito.de
Scan Status Ok
Last Scan2024-10-28T04:19:40+00:00
Next Scan 2024-11-27T04:19:40+00:00

Last Scan

Scanned2024-10-28T04:19:40+00:00
URL https://nivito.de/robots.txt
Redirect https://www.nivito.de/robots.txt
Redirect Domain www.nivito.de
Redirect Base nivito.de
Domain IPs 104.21.61.246, 172.67.217.60, 2606:4700:3034::ac43:d93c, 2606:4700:3035::6815:3df6
Redirect IPs 104.21.61.246, 172.67.217.60, 2606:4700:3034::ac43:d93c, 2606:4700:3035::6815:3df6
Response IP 104.21.61.246
Found Yes
Hash 5cbd926549e48bfde8900d94188a2e984f42e9c17d7b62c9568afc22c5d211e9
SimHash 4a36ec40cc82

Groups

*

Rule Path
Disallow /

bingbot
googlebot

Rule Path
Disallow

gigabot

Rule Path
Disallow /

robozilla

Rule Path
Disallow /

nutch

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

bspider

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

sogou

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

xyz

Rule Path
Disallow /

bl.uk_lddc_bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

nimbostratus-bot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

*

Rule Path
Disallow /nogooglebot/
Disallow /app/
Disallow /.github/
Disallow /bin/
Disallow /db-backup/
Disallow /dev/
Disallow /generated/
Disallow /lib/
Disallow /phpserver/
Disallow /app/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /vendor/

Other Records

Field Value
crawl-delay 120

Other Records

Field Value
sitemap https://www.nivito.de/pub/sitemap/de_DE.xml