Log In Sign Up

isoladellapoesia.com
robots.txt

Robots Exclusion Standard data for isoladellapoesia.com

Resource Scan

Scan Details

Site Domain isoladellapoesia.com
Base Domain isoladellapoesia.com
Scan Status Ok
Last Scan2025-03-22T09:21:46+00:00
Next Scan 2025-04-21T09:21:46+00:00

Last Scan

Scanned2025-03-22T09:21:46+00:00
URL https://isoladellapoesia.com/robots.txt
Redirect https://www.isoladellapoesia.com/robots.txt
Redirect Domain www.isoladellapoesia.com
Redirect Base isoladellapoesia.com
Domain IPs 195.130.247.171
Redirect IPs 195.130.247.171
Response IP 195.130.247.171
Found Yes
Hash 280fe06014b67e536d7aac639551d2976f2ac6f04d085cde31a5e3435dc32e0b
SimHash f234a1e6c9b9

Groups

*

Rule Path
Disallow /regolamento.php
Disallow /block.php
Disallow /robots.txt
Disallow /404.txt

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 800

a6-indexer

Rule Path
Disallow /

admantx

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /
Disallow /

web-archive-it.com

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /regolamento.php
Disallow /chisono.php
Disallow /webmaster.php

yasni

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

webcitation

Rule Path
Disallow /

webcite

Rule Path
Disallow /

atomz/1.0

Rule Path
Disallow /

mozilla/5.0 (x11; u; linux x86_64; en-us; rv:1.9.0.19; aggregator:spinn3r (spinn3r 3.1); http://spinn3r.com/robot) gecko/2010040121 firefox/3.0.19

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mj12bot/v1.4.0

Rule Path
Disallow /

mj12bot/v1.2.4

Rule Path
Disallow /

mj12bot/v1.2.3

Rule Path
Disallow /

mj12bot/v1.0.8

Rule Path
Disallow /

mj12bot/v1.0.7

Rule Path
Disallow /

mj12bot/v1.0.6

Rule Path
Disallow /

mj12bot/v1.0.5

Rule Path
Disallow /

contextad bot 1.0

Rule Path
Disallow

yahoo pipes 1.0

Rule Path
Disallow /

yahoo pipes 2.0

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

screenerbot

Rule Path
Disallow /

hubspot webcrawler
meanpathbot/1.0; (+http://www.meanpath.com/meanpathbot.html)

Rule Path
Disallow /

*

Rule Path
Disallow /blog/feed/

admantx platform semantic analyzer

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

nutch-1.4/nutch-1.4

Rule Path
Disallow /

nutch-1.8/nutch-1.8

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

a6-indexer

Rule Path
Disallow /

whoismindbot

Rule Path
Disallow /

whoismindbot/1.0

Rule Path
Disallow /

whoismindbot/1.1

Rule Path
Disallow /

whoismindbot/2.0

Rule Path
Disallow /

black hole
cherrypicker
disco pump 3.1
emailcollector
emailsiphon
emailwolf
extractorpro

Rule Path
Disallow /

teleport
backdoorbot/1.0
offline explorer
superbot
superbot/2.6
teleport
titan
webbandit
webcopier
webcopy
webmirror
webreaper
website extractor
winhttrack
news bot
news bot /2.1
ezooms
ezooms/1.0
mail.ru_bot/2.0
j12bot/v1.4.3
acoonbot
acoonbot/4.11.1
exabot
exabot/3.0
obot
obot/2.3.1
acoonbot
ahrefsbot
diffbot
diffbot/0.1
spinn3r
spinn3r 3.1

Rule Path
Disallow /

moget
ichiro

Rule Path
Disallow /

naverbot
yeti

Rule Path
Disallow /

baiduspider
baiduspider-video
baiduspider-image
sogou spider
youdaobot

Rule Path
Disallow /

yandex
yandexbot
yandexbot/3.0

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webcopy

Rule Path
Disallow /

web image collector

Rule Path
Disallow /

webmirror

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

lexibot

Rule Path
Disallow /

website extractor

Rule Path
Disallow /

winhttrack

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

teleport

Rule Path
Disallow /

hloader

Rule Path
Disallow /

netattache

Rule Path
Disallow /

netattache light 1.1

Rule Path
Disallow /

netmechanic

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /
Back to top

Comments

  • https://www.facebook.com/externalhit_uatext.php
  • User-agent: facebookexternalhit
  • Sogou
  • A6-Indexer
  • SPIDER MAIL
  • MOTORI DI RICERCA GIAPPONESI
  • MOTORI DI RICERCA KOREANI
  • MOTORI DI RICERCA CINESI
  • MOTORI DI RICERCA RUSSI
Back to top

Warnings

  • 2 invalid lines.
Back to top