nivito.se
robots.txt

Robots Exclusion Standard data for nivito.se

Resource Scan

Scan Details

Site Domain nivito.se
Base Domain nivito.se
Scan Status Ok
Last Scan2024-11-16T04:31:24+00:00
Next Scan 2024-12-16T04:31:24+00:00

Last Scan

Scanned2024-11-16T04:31:24+00:00
URL https://nivito.se/robots.txt
Redirect https://www.nivito.se/robots.txt
Redirect Domain www.nivito.se
Redirect Base nivito.se
Domain IPs 34.159.81.44
Redirect IPs 34.159.81.44
Response IP 34.159.81.44
Found Yes
Hash bab1007ee2a8636e725bb5a37e05e800b3b75c8a7a4b6975571f6221363fda14
SimHash 4b32cc40c892

Groups

*

Rule Path
Disallow /

bingbot
googlebot

Rule Path
Disallow

gigabot

Rule Path
Disallow /

robozilla

Rule Path
Disallow /

nutch

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

bspider

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

sogou

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

xyz

Rule Path
Disallow /

bl.uk_lddc_bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

nimbostratus-bot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

*

Rule Path
Disallow /nogooglebot/
Disallow /app/
Disallow /.github/
Disallow /bin/
Disallow /db-backup/
Disallow /dev/
Disallow /generated/
Disallow /lib/
Disallow /phpserver/
Disallow /app/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /vendor/

Other Records

Field Value
crawl-delay 120

Other Records

Field Value
sitemap https://www.nivito.se/pub/sitemap/sv_SE.xml