Log In Sign Up

sifar.it
robots.txt

Robots Exclusion Standard data for sifar.it

Resource Scan

Scan Details

Site Domain sifar.it
Base Domain sifar.it
Scan Status Ok
Last Scan2024-09-27T00:03:58+00:00
Next Scan 2024-10-27T00:03:58+00:00

Last Scan

Scanned2024-09-27T00:03:58+00:00
URL https://sifar.it/robots.txt
Redirect https://www.sifar.it/robots.txt
Redirect Domain www.sifar.it
Redirect Base sifar.it
Domain IPs 93.186.249.9
Redirect IPs 93.186.249.9
Response IP 93.186.249.9
Found Yes
Hash e658e9dadaf164ff8247112d77b47d4e828af006951e54b20ea82d2b9203f886
SimHash c05661da410a

Groups

gptbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

winhttrack

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

vscooter

Rule Path
Disallow /

psbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

yandex

Rule Path
Disallow /

taptubot

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

infopath

Rule Path
Disallow /

infopath.2

Rule Path
Disallow /

swebot

Rule Path
Disallow /

ec2linkfinder

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

searchmetericsbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

ip-web-crawler.com

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

aboundex

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

spbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

a6-indexer

Rule Path
Disallow /

riddler

Rule Path
Disallow /

loadtimebot

Rule Path
Disallow /

obot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

advbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

lssrocketcrawler

Rule Path
Disallow /

gsa-crawler

Rule Path
Disallow /

nutch

Rule Path
Disallow /

tbot-nutch

Rule Path
Disallow /

thunderstone

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

ranksonicbot

Rule Path
Disallow /

betabot

Rule Path
Disallow /

parsijoo-bot

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

gocrawl

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

applebot

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

rankactivelinkbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

seeker

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

yoozbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

seekport

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

*
*

Rule Path
Disallow /org/
Back to top

Other Records

Field Value
sitemap https://www.sifar.it/sitemap.xml
Back to top

Comments

  • Bloccati la maggior parte degli spider conosciuti
  • GPTBot openAI
  • MOTORI DI RICERCA GIAPPONESI
  • MOTORI DI RICERCA KOREANI
  • Gia presenti su sifar
  • presi da http://smythies.com (alcuni gia presenti su sifar)
  • User-agent: Googlebot-Image
  • Disallow: /
Back to top

Warnings

  • 2 invalid lines.
  • `visit-time` is not a known field.
Back to top