nivito.it
robots.txt

Robots Exclusion Standard data for nivito.it

Resource Scan

Scan Details

Site Domain nivito.it
Base Domain nivito.it
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-11-07T06:38:00+00:00
Next Scan 2025-01-06T06:38:00+00:00

Last Successful Scan

Scanned2024-08-17T09:29:09+00:00
URL https://nivito.it/robots.txt
Redirect https://www.nivito.it/robots.txt
Redirect Domain www.nivito.it
Redirect Base nivito.it
Domain IPs 34.159.239.254
Redirect IPs 34.159.239.254
Response IP 34.159.239.254
Found Yes
Hash f431b140107d703e8c874cf4f019658479bb5c544e9f421084f4d4dee26bdeb5
SimHash 0b32c440c892

Groups

gigabot

Rule Path
Disallow /

robozilla

Rule Path
Disallow /

nutch

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

bspider

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

sogou

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

xyz

Rule Path
Disallow /

bl.uk_lddc_bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

nimbostratus-bot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

*

Rule Path
Disallow /nogooglebot/
Disallow /app/
Disallow /.github/
Disallow /bin/
Disallow /db-backup/
Disallow /dev/
Disallow /generated/
Disallow /lib/
Disallow /phpserver/
Disallow /app/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /vendor/

Other Records

Field Value
crawl-delay 120

Other Records

Field Value
sitemap https://www.nivito.it/pub/sitemap/it_IT.xml