Log In Sign Up

gustoegusti.it
robots.txt

Robots Exclusion Standard data for gustoegusti.it

Resource Scan

Scan Details

Site Domain gustoegusti.it
Base Domain gustoegusti.it
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-10-26T12:11:10+00:00
Next Scan 2025-01-24T12:11:10+00:00

Last Successful Scan

Scanned2024-05-31T12:08:20+00:00
URL https://gustoegusti.it/robots.txt
Domain IPs 104.21.9.106, 172.67.159.191, 2606:4700:3033::6815:96a, 2606:4700:3035::ac43:9fbf
Response IP 172.67.159.191
Found Yes
Hash 00d2d95bddaba34d884477e957d0d55ad7672164cf4c171dd7ed42d40b92b736
SimHash b2de71408863

Groups

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

maxthon

Rule Path
Disallow /

cncdialer

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

exabot

Rule Path
Disallow /

moreoverbot

Rule Path
Disallow /

cyberalert

Rule Path
Disallow /

newscan

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

lexxebot/1.0

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

sitebot/0.1

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

netseer crawler

Rule Path
Disallow /

trovitbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

discobot

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

nerdbynature.bot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

addthis.com

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

bl.uk_lddc_bot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

bender

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

yasni

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

integromedb

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bdcbot

Rule Path
Disallow /

wesee:search

Rule Path
Disallow /

admantx

Rule Path
Disallow /

spbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

badbot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

arachnophilia

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

webcopy

Rule Path
Disallow /

*

Rule Path
Disallow /cdn-cgi/
Back to top

Comments

  • Agentes bloqueados por idioma
  • Agentes nocivos
  • Agentes cloudflare
Back to top

Warnings

  • 2 invalid lines.
Back to top