Log In Sign Up

miwo.it
robots.txt

Robots Exclusion Standard data for miwo.it

Resource Scan

Scan Details

Site Domain miwo.it
Base Domain miwo.it
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-10-05T15:02:49+00:00
Next Scan 2024-11-04T15:02:49+00:00

Last Successful Scan

Scanned2024-08-14T07:19:55+00:00
URL https://miwo.it/robots.txt
Domain IPs 80.88.86.105
Response IP 80.88.86.105
Found Yes
Hash 77eb855688b9b0129c16799d3d1a6ec302f34cf2188701bc027d3370076429f4
SimHash 4096e1d64908

Groups

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

winhttrack

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

vscooter

Rule Path
Disallow /

psbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

yandex

Rule Path
Disallow /

taptubot

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

infopath

Rule Path
Disallow /

infopath.2

Rule Path
Disallow /

swebot

Rule Path
Disallow /

ec2linkfinder

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

searchmetericsbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

ip-web-crawler.com

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

aboundex

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

spbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

a6-indexer

Rule Path
Disallow /

riddler

Rule Path
Disallow /

loadtimebot

Rule Path
Disallow /

obot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

advbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

lssrocketcrawler

Rule Path
Disallow /

gsa-crawler

Rule Path
Disallow /

nutch

Rule Path
Disallow /

tbot-nutch

Rule Path
Disallow /

thunderstone

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

ranksonicbot

Rule Path
Disallow /

betabot

Rule Path
Disallow /

parsijoo-bot

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

gocrawl

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

applebot

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

rankactivelinkbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

seeker

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

yoozbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

seekport

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

*
*

Rule Path
Disallow /log/
Back to top

Other Records

Field Value
sitemap http://www.miwo.it/sitemap.xml
sitemap https://www.miwo.it/sitemap.xml
Back to top

Comments

  • Bloccati la maggior parte degli spider conosciuti
  • MOTORI DI RICERCA GIAPPONESI
  • MOTORI DI RICERCA KOREANI
  • presi da http://smythies.com
  • User-agent: Googlebot-Image
  • Disallow: /
Back to top

Warnings

  • 3 invalid lines.
  • `visit-time` is not a known field.
Back to top